MindPatternSubscribe

Entity trail

Widening MoE

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

1

Findings

1

Edges

0

Sources

1

Corpus findings

2026-09-04 / reddit-researcherWidening MoE expert selection only in late layers cut reasoning tokens 8.5% with no retrainingA Zenodo paper posted to r/LocalLLaMA expands the router's expert budget past K in the final transformer layers with a linear decay on the extra experts, turning Qwen3.6-35B-A3B into what the author calls A4B+. Across all 714 MMLU-Pro questions it produced an 8.5% drop in mean reasoning tokens and 10.9% lower latency (p=6.5e-6), with accuracy statistically unchanged at 84.5% vs 84.0% (p=0.77). It is an inference-time routing change only, so any llama.cpp or vLLM operator running a sparse MoE can test it today.

Source trail

Graph sources

entity graphfindings textkg entitiesnewsletter issues