A 1-bit 27B model hits 25-30 tokens a second in Chrome

A developer running as mentria.ai built a browser inference engine from scratch in WebGPU and WGSL, then used it to run Prism ML's Bonsai-27B, a model natively trained at 1-bit weights, on an RTX 3060 Laptop GPU. No local install, nothing sent off the machine. The result: 25 to 30 tokens a second, posted to r/LocalLLaMA on September 10.

The compression is the part that makes a 6 GB laptop GPU plausible at all. Bonsai-27B stores one sign bit per weight plus one FP16 scale per 128 weights, averaging about 1.14 bits per parameter. That puts the entire 27B model at 3.8 GB, small enough to sit inside a consumer laptop's VRAM budget alongside Chrome itself.

Speed moved fast even after that. The same setup decoded at 15 tokens a second two days earlier, so the author roughly doubled throughput in 48 hours. Each token costs 804 GPU dispatches, 401 of them the 1-bit matrix-vector kernel that does the actual weight lookups.

One thing to flag: the author ran his own eval deltas rather than reporting Prism ML's published benchmarks, so the quality tradeoff of running a model this compressed isn't independently confirmed here. Speed and memory footprint are measured; output quality is the author's own read.

Watch whether other WebGPU inference projects pick up 1-bit-native models next, since the bottleneck this result removes isn't the browser runtime, it's model availability at this bit width.