Infra2026-09-23 · source-backed

Disaggregated quantization lifted 1-bit Qwen3.8-27B by 32.5 points on MMLU-Pro.

Issue

2026-09-23

Confidence

source-backed

Sources

Redaction

redacted

Story

arXiv 2609.26333 trains a separate NVFP4 prefill checkpoint while decode keeps compact weights for memory traffic. On released GGUF decoders that's 32.5 points on MMLU-Pro and 35.3 on MMMU-Pro without touching the decode checkpoint, plus a 1.78x TTFT speedup at 8K prompts in llama.cpp by streaming prefill weights off SSD. Prefill and decode no longer have to share one quantization compromise, which is the most useful local-inference result of the week.

Source trail

arXiv 2609.26333 More from arxiv.org

Entities

Disaggregated GGUF MMLU MMMU NVFP4 Prefill Qwen3 SSD TTFT

Provenance

Canonical issue: Ramsay Research Agent — September 23, 2026
AI generated: no
Story unit: 2026-09-23-disaggregated-quantization-lifted-1-bit-qwen3-8-27b-by-32-5-points-on-mmlu-pro
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

Disaggregated quantization lifted 1-bit Qwen3.8-27B by 32.5 points on MMLU-Pro.

Story

Related stories

Heretic's name is turning up inside Hugging Face model IDs

CachyLLama's disk-backed KV cache cuts agentic time-to-first-token from 143 seconds to 0.99.

llama.cpp Merged stdio MCP Support. Your Local Model Can Spawn Tool Servers Now.

Edge0 runs a 35B MoE model in under 3GB of RAM

Transformers can load a llama.cpp GGUF file directly and approach llama.cpp throughput on Apple silicon.

bartowski measured which tensors actually break under quantization, and Q3_K_M got 15% smaller.

Source trail

Entities

Provenance

Disaggregated quantization lifted 1-bit Qwen3.8-27B by 32.5 points on MMLU-Pro.

Story

Related stories

Heretic's name is turning up inside Hugging Face model IDs

CachyLLama's disk-backed KV cache cuts agentic time-to-first-token from 143 seconds to 0.99.

llama.cpp Merged stdio MCP Support. Your Local Model Can Spawn Tool Servers Now.

Edge0 runs a 35B MoE model in under 3GB of RAM

Transformers can load a llama.cpp GGUF file directly and approach llama.cpp throughput on Apple silicon.

bartowski measured which tensors actually break under quantization, and Q3_K_M got 15% smaller.

Source trail

Entities

Provenance

Story

Related stories

Heretic's name is turning up inside Hugging Face model IDs

CachyLLama's disk-backed KV cache cuts agentic time-to-first-token from 143 seconds to 0.99.

llama.cpp Merged stdio MCP Support. Your Local Model Can Spawn Tool Servers Now.

Edge0 runs a 35B MoE model in under 3GB of RAM

Transformers can load a llama.cpp GGUF file directly and approach llama.cpp throughput on Apple silicon.

bartowski measured which tensors actually break under quantization, and Q3_K_M got 15% smaller.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance

Story

Related stories

Heretic's name is turning up inside Hugging Face model IDs

CachyLLama's disk-backed KV cache cuts agentic time-to-first-token from 143 seconds to 0.99.

llama.cpp Merged stdio MCP Support. Your Local Model Can Spawn Tool Servers Now.

Edge0 runs a 35B MoE model in under 3GB of RAM

Transformers can load a llama.cpp GGUF file directly and approach llama.cpp throughput on Apple silicon.

bartowski measured which tensors actually break under quantization, and Q3_K_M got 15% smaller.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance