Gemma 4 Produced a Working Pac-Man in 6,209 Tokens. Qwen 3.6 Used 33,946 Tokens and Failed.

A head-to-head test on an M5 Max MacBook Pro (64GB RAM) put Gemma 4 31B against Qwen 3.6 27B on a practical task: build a Pac-Man game. Gemma finished in 3 minutes 51 seconds, used 6,209 tokens, and produced a working game. Qwen took 18 minutes 4 seconds, burned through 33,946 tokens, and the game didn't work.

Despite lower raw token throughput (27 tok/s vs 32 tok/s), Gemma's dramatically better code efficiency made it faster in wall-clock time and produced a functional result. Five times fewer tokens. Five times faster. Actually works versus doesn't.

This challenges the assumption that bigger context and faster inference speed are what matter for practical coding. They're not. Token efficiency, how much useful work gets done per token, is what determines whether you get a working product. A model that writes tight, correct code in 6K tokens beats a model that rambles for 34K tokens and still fails.

The connection to rtk (story #1) is direct. Token efficiency is the emerging meta for AI-assisted development. At the infrastructure level, rtk compresses what goes into the context. At the model level, Gemma 4 compresses what comes out. Both attacks on the same problem: making every token count.

For anyone doing local model development, especially game dev or creative coding, Gemma 4 31B deserves a serious look. The 482 upvotes and 114 comments on r/LocalLLaMA suggest the community agrees. And with AMD's Halo Box approaching launch with 128GB unified memory, running 31B models locally is about to get a lot more accessible.

The practical takeaway: don't evaluate models on benchmarks alone. Run your actual task. Measure tokens consumed versus quality of output. The model that uses fewer tokens to produce working code is the better model, regardless of what the leaderboard says.

Source: r/LocalLLaMA