Fetching from the wire…
Public story · 2026-09-08 · high
The gap over a byte-matched quant grows from 5 points at 27B to nearly 20 points at 4B, and coding breaks it.
Why now: The coding warning was added to the post only after testers ran into repetition loops on code.
A task-tuned quant holds Qwen3.8-27B at 82.81% held-out reasoning accuracy, just under BF16's 83.59%, at 15% of the original size, per a TAK post to r/LocalLLaMA.
Against a byte-matched Unsloth UD IQ2_S quant at the same size, TAK's accuracy edge is 5.47 points at 27B. That gap widens as models shrink: +11.72 points on Qwen3.5-4B, +14.06 on Gemma 4 E4B, +19.53 on Gemma 3 4B QAT.
The method builds an importance matrix from a corpus matched to the target task, then finds the smallest size before the model's output collapses. From there it trades precision between tensors inside a fixed byte budget, no pruning, no fine-tuning, no merging. It moves bits to where the calibration data says they're needed.
The author added a warning after the post went up. The reasoning-specialized quant breaks down on coding, where testers hit repetition loops. That's a real limit. The quant is task-specific, not a general swap-in for full precision.
Each link below shares sources, entities, or timing with this story.
The models are good. The license is the real story. Google released Gemma 4 on April 2 with four variants: E2B, E4B, 26B MoE, and 31B Dense. All built on the Gemini 3 architecture. The 31B Dense variant claimed #3 on Arena AI's text leaderboard, beating models 20x its size. Th...
Google DeepMind released Gemma 4 on April 2 with four model sizes (E2B, E4B, 26B MoE, 31B Dense) under Apache 2.0. Multimodal (text, vision, audio). 256K context. Native thinking and tool-calling optimized for agentic workflows. Day-zero ecosystem support across vLLM, llama.cp...
One file handles tokenizer, transformer, KV cache, sampling and CPU kernels with no external library doing the interesting parts, producing about a 5.0 GB model file (GitHub). Weights are int8 with FP16 scales, linear-layer inputs dynamically quantized to int8, other activatio...
Google dropped Gemma 4 and it's not incremental. The 31B dense model ranks #3 on Arena AI with an ELO of 1,452, scores 85.2% on MMLU Pro, 89.2% on AIME 2026, and 80.0% on LiveCodeBench v6. It outperforms models 20x its size. Under Apache 2.0. At $0.20 per run. Only Opus 4.6 an...
Google DeepMind shipped Quantization-Aware Training checkpoints for every Gemma 4 size, and the headline number is genuinely useful: the smallest model goes from 11.4GB to 1.1GB. That's 0.84GB if you go text-only. Up to ~72% lower VRAM and 2x faster inference on mobile NPUs, w...
Alibaba released Qwen3.6-27B on April 22. Dense architecture. Open weights. 77.2% on SWE-bench Verified, within 3.7 points of Claude Opus 4.6. On SkillsBench, it scores 48.2% versus its own 397B MoE predecessor's 30.0%. That's a 77% improvement with 14.8x fewer parameters. Let...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.