llama.cpp PR #21038: Hadamard Rotation Sharply Improves Low-Bit KV-Cache Quality

GitHub (ggml-org/llama.cpp)·medium signal

A llama.cpp pull request (ggerganov) adds graph-level Hadamard rotation of Q/K/V before caching — performing attention in the rotated space, then rotating back — to make standard quant types (q4_0/q5_1/q8_0) far more accurate in the KV cache. Reported benchmarks are dramatic: Qwen3 0.6B q5_1 KV perplexity drops from ~61.7 to ~14.1, and q4_1 from ~212 to ~22.3. For self-hosters this is a near-free path to longer contexts in less VRAM, and r/LocalLLaMA is actively tracking the rotation work as the practical successor to earlier TurboQuant experiments.