Infra2026-06-13 · source-backed

llama.cpp just made low-bit KV cache dramatically more accurate, nearly for free.

Issue

2026-06-13

Confidence

source-backed

Sources

Redaction

passed

Story

PR #21038 adds graph-level Hadamard rotation of Q/K/V before caching, doing attention in the rotated space then rotating back, which makes standard quant types far more accurate in the KV cache. The benchmarks are not subtle: Qwen3 0.6B q5_1 KV perplexity drops from ~61.7 to ~14.1, and q4_1 from ~212 to ~22.3. For self-hosters this is a near-free path to longer contexts in less VRAM, and r/LocalLLaMA is tracking it as the successor to earlier TurboQuant work.

Source trail

PR #21038

Entities

Hadamard LocalLLaMA Qwen3 TurboQuant VRAM

Provenance

Canonical issue: Ramsay Research Agent — June 13, 2026
AI generated: no
Story unit: 2026-06-13-llama-cpp-just-made-low-bit-kv-cache-dramatically-more-accurate-nearly-for-free
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

llama.cpp just made low-bit KV cache dramatically more accurate, nearly for free.

Story

Related stories

Google Ships Gemma 4 Under Apache 2.0. Four Models, Arena #3, and the License That Changes Everything.

Gemma 4 Drops Four Open Models, and the 31B Dense Beats a 397B MoE

Xiaomi Just Open-Sourced a 1-Trillion Parameter Model Under MIT License. It Built a Compiler in 4 Hours.

r/LocalLLaMA discovered the EU transparency code and realized watermarking may reach open weights.

Qwen jumped from 1% to 69% of open-model fine-tunes

llama.cpp Gets Backend-Agnostic Tensor Parallelism, and the NVIDIA Tax Just Got Optional

Google open-sourced DiffusionGemma, a text-diffusion model that generates 4x faster, and you can run it locally today

Gemma 4 QAT checkpoints drop the smallest model from 11.4GB to 1.1GB

Source trail

Entities

Provenance

llama.cpp just made low-bit KV cache dramatically more accurate, nearly for free.

Story

Related stories

Google Ships Gemma 4 Under Apache 2.0. Four Models, Arena #3, and the License That Changes Everything.

Gemma 4 Drops Four Open Models, and the 31B Dense Beats a 397B MoE

Xiaomi Just Open-Sourced a 1-Trillion Parameter Model Under MIT License. It Built a Compiler in 4 Hours.

r/LocalLLaMA discovered the EU transparency code and realized watermarking may reach open weights.

Qwen jumped from 1% to 69% of open-model fine-tunes

llama.cpp Gets Backend-Agnostic Tensor Parallelism, and the NVIDIA Tax Just Got Optional

Google open-sourced DiffusionGemma, a text-diffusion model that generates 4x faster, and you can run it locally today

Gemma 4 QAT checkpoints drop the smallest model from 11.4GB to 1.1GB

Source trail

Entities

Provenance