A 35 billion parameter model fits in 3GB of RAM

Edge0-35B-A3B squeezes a 35 billion parameter model into about 3 GiB of memory by pulling only its active experts from SSD. That fits on a Mac mini because the model never needs its full weight set in RAM at once.

The model is built on Qwen3.5-MoE and fires just 4 of its 256 experts per token. The runtime fetches only those weights from storage on demand, so it never holds all 256 experts resident in memory. That keeps peak active memory near 2.9 to 3 GiB.

Reading from SSD only works if it doesn't stall generation. A trained prerouter predicts which experts the next step will need, so the storage read overlaps with compute and doesn't block it. That overlap alone adds up to 59% to decode throughput.

On a 24GB Mac mini M4 Pro, the model decodes at about 20 tokens a second and prefills at 113 to 140 tokens a second.

Running the model in int4 costs accuracy, and the paper doesn't hide it. A distillation pass called Recover-LoRA, trained on the frozen int4 base, closes most of that gap but leaves a reported 3.9 point deficit against full precision.