Widening MoE expert selection only in late layers cut reasoning tokens 8.5% with no retraining
Zenodo (via r/LocalLLaMA, 122 upvotes)·medium signal
A Zenodo paper posted to r/LocalLLaMA expands the router's expert budget past K in the final transformer layers with a linear decay on the extra experts, turning Qwen3.6-35B-A3B into what the author calls A4B+. Across all 714 MMLU-Pro questions it produced an 8.5% drop in mean reasoning tokens and 10.9% lower latency (p=6.5e-6), with accuracy statistically unchanged at 84.5% vs 84.0% (p=0.77). It is an inference-time routing change only, so any llama.cpp or vLLM operator running a sparse MoE can test it today.