Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-07 · high
The code streams dormant experts off a 1.7 terabyte NVMe drive to hit an 8.24 GB RAM peak, at 32 seconds per token.
Why now: This is landing now because the repo went from zero to 3,009 stars in six days, about 500 a day, unusually fast growth for a systems trick rather than a new model.
FareedKhan ran Kimi K3, a 2.78-trillion-parameter model, on a single CPU using 8.24 GB of RAM, per the project's GitHub repo.
Loaded the normal way, that model needs about 5,560 GB of RAM, more than any single machine holds. Anyone trying to run something this size without a GPU cluster hits the same wall now: storage speed, not compute, decides whether it's possible.
The code streams the dormant mixture-of-experts layers off an NVMe drive and keeps only the active trunk in memory. Output stays byte-identical whether you give it 8 GB of RAM or 224 GB, according to the repo.
Routed experts ship pre-quantized to 0.53125 bytes per parameter, 4-bit with a shared E8M0 scale. The dense trunk stays in bf16. All of it runs in 700 lines of C99.
None of it is fast. A laptop preset runs about 32 seconds per token. A 128 GB server, an AMD EPYC 7763 with 3.2 TB of NVMe, gets that down to 19 to 21 seconds per token. Holding the weights on disk still takes roughly 1.7 TB of storage.
This isn't a way to serve a trillion-parameter model to real users. It's a demonstration of where the wall sits for anyone trying. The 32-second-per-token cost tracks NVMe read speed, not GPU speed, so that's the number that has to move before any of this gets practical.
Each link below shares sources, entities, or timing with this story.
Shared entities / Same source domain / Shared topic / Earlier coverage
Both cover MoE, NVMe, RAM; reported by the same outlet (github.com); overlapping topics (cost, dense, expert, nvme, token).
Shared entities / Same source
Both cover C99, FareedKhan, MoE, NVMe; cite the same source (FareedKhan-dev/kimi-k3-in-c).
Shared entities / Same source / Earlier coverage
Both cover C99, FareedKhan, NVMe; cite the same source (FareedKhan-dev/kimi-k3-in-c); earlier C99 coverage from 2026-08-03.
Shared entities / Same source domain / Shared topic / Earlier coverage
Both cover NVMe, RAM; reported by the same outlet (github.com); overlapping topics (bandwidth, expert, nvme, token).
Shared entities / Shared topic / Earlier coverage
Both cover Cost, FLOPs, MoE; overlapping topics (active, cost, token); earlier Cost coverage from 2026-04-26.
Both cover MoE, RAM; overlapping topics (active, dense, expert, token); earlier MoE coverage from 2026-08-04.
Shared entities / Same source domain / Shared topic / Earlier coverage
Both cover MoE, RAM; reported by the same outlet (github.com); overlapping topics (expert, token).
Shared entity: MoE / Same source domain / Shared topic / Earlier coverage
Both cover MoE; reported by the same outlet (github.com); overlapping topics (bandwidth, days, expert).