Fetching from the wire…
01
02
03
04
05
06
07
08
Models2026-09-04 · source-backed
A Zenodo paper expands the router's expert budget past K in the final transformer layers with a linear decay on the extra experts, turning Qwen3.6-35B-A3B into what the author calls A4B+. Across all 714 MMLU-Pro questions: 8.5% drop in mean reasoning tokens, 10.9% lower latency (p=6.5e-6), accuracy statistically unchanged at 84.5% against 84.0% (p=0.77). Inference-time routing change only, so any llama.cpp or vLLM operator running a sparse MoE can test it today. Zenodo
Each link below shares sources, entities, or timing with this story.
Alibaba's 9B model outperforms OpenAI's 120B on GPQA Diamond (81.7 vs 71.5), MMLU-Pro (82.5 vs 80.8), and multilingual MMMLU. Uses hybrid Gated Delta Network + sparse MoE with 262K native context. Apache 2.0 on HuggingFace. VentureBeat
SpeakoFlow Mini fine-tunes Qwen3.5-0.8B to apply only the corrections a speaker actually made and leave the rest alone. On the author's English-only benchmark it scored 70.7% against GPT-5.6 Luna's 65.0% under the same fixed short prompt with reasoning disabled, but the 95% in...
Three points behind GLM-5.3 at 60, tying GPT-5.6 Terra and Muse Spark 1.2, at $0.09 per task against $0.68 for GLM-5.3 max (Latent Space). It burned 149M output tokens to run the index, of which 134M were reasoning tokens, more than Kimi K3 at 133M or Qwen3.8 2.4T A95B at 136M...
The architecture report describes a 125B sparse MoE activating 6B parameters per token, plus 51B of n-gram embedding tables held off the accelerator in host memory with prefetching (arXiv 2608.30320). Against the prior 397B-A17B model it leads on 8 of 14 pre-training benchmark...
An r/LocalLLaMA post (215 upvotes) ran identical inputs through Qwen 35B-A3B and Gemma 26B-A4B and found the tokenizers diverge almost entirely on code: 2.6x apart on HTML/JS, but 1,025 vs 1,039 tokens on a 55-line instruction document. Near-identical on prose. That's a near-3...
Measuring expert time on two datacenter GPU generations shows it is linear in neither token count (EPLB, LPLB, UltraEP) nor activated-expert count (METRO). Below roughly 156 to 168 tokens, HBM weight streaming dominates so cost attaches to activated replicas. Above it, grouped...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.