TurboFieldfare Runs Gemma 4 26B in ~2 GB of RAM on Any M-Series Mac by Streaming Experts From SSD With pread, Not mmap
Andrey Mikhaylov's Swift and Metal runtime keeps only Gemma 4 26B-A4B's 1.35 GB shared core plus an FP16 KV cache resident and streams each token's needed experts off SSD, fitting the 14.3 GB model into roughly 2 GB and running on 8 GB machines. Measured throughput is 5.1–6.3 tok/s on an M2 MacBook Air and 31–35 tok/s on an M5 Pro, and in the Hacker News thread (833 points, 295 comments, #1 on July 29) the author reports cold expert loads at 2.8 ms via pread versus 10 ms via mmap — the specific detail that makes the approach work. Apache 2.0, 1.7k stars, CLI and native Mac app. This is a different animal from the Kimi K3 HTTP-expert-streaming hack covered on July 29: local SSD rather than network, a purpose-built Metal runtime rather than a proof of concept, and it targets a model a solo builder would actually keep resident.
↳ Follow the thread