FareedKhan Squeezed 2.78 Trillion Parameters Onto One CPU

FareedKhan ran Kimi K3, a 2.78-trillion-parameter model, on a single CPU using 8.24 GB of RAM, per the project's GitHub repo.

Loaded the normal way, that model needs about 5,560 GB of RAM, more than any single machine holds. Anyone trying to run something this size without a GPU cluster hits the same wall now: storage speed, not compute, decides whether it's possible.

The code streams the dormant mixture-of-experts layers off an NVMe drive and keeps only the active trunk in memory. Output stays byte-identical whether you give it 8 GB of RAM or 224 GB, according to the repo.

Routed experts ship pre-quantized to 0.53125 bytes per parameter, 4-bit with a shared E8M0 scale. The dense trunk stays in bf16. All of it runs in 700 lines of C99.

None of it is fast. A laptop preset runs about 32 seconds per token. A 128 GB server, an AMD EPYC 7763 with 3.2 TB of NVMe, gets that down to 19 to 21 seconds per token. Holding the weights on disk still takes roughly 1.7 TB of storage.

What it actually proves

This isn't a way to serve a trillion-parameter model to real users. It's a demonstration of where the wall sits for anyone trying. The 32-second-per-token cost tracks NVMe read speed, not GPU speed, so that's the number that has to move before any of this gets practical.