TurboFieldfare Fits a 26B Model Into 2GB of RAM

TurboFieldfare runs Gemma 4 26B in about 2 GB of RAM on any M-series Mac, according to the project's GitHub repo. The model itself is 14.3 GB, and the runtime works on machines with as little as 8 GB of total memory. That matters because RAM has been the ceiling on running large models locally on a Mac.

The runtime keeps only Gemma's 1.35 GB shared core and an FP16 key-value cache resident in memory. It streams each token's needed experts off SSD instead.

Gemma 4 26B-A4B is a mixture-of-experts model, so most of its 26 billion parameters sit idle for any single token. There's no need to hold all of them in RAM at once.

The unlock is how TurboFieldfare reads from disk. Mikhaylov reports cold expert loads take 2.8 milliseconds via pread, compared with 10 milliseconds via mmap. He calls that gap the reason per-token expert swapping is fast enough to use.

The thread hit number one on Hacker News on July 29, with 833 points. Throughput still depends on the chip: 5.1 to 6.3 tokens per second on an M2 MacBook Air, 31 to 35 on an M5 Pro.

The Air numbers work for chat, not much beyond that. TurboFieldfare is Apache 2.0 licensed with 1.7k GitHub stars, shipping a CLI and a native Mac app.

The bigger claim: this breaks the rule that RAM must scale with a model's total parameters, at least for mixture-of-experts models on Apple Silicon. Worth watching whether other local-inference runtimes copy the same read pattern.