llama.cpp KV Rotation PR Shows Q8 Quantization Tanks AIME25 — But Rotation Mostly Recovers It
r/LocalLLaMA / GitHub·medium signal
A llama.cpp pull request (#21038) by ggerganov reveals that existing Q8_0 KV cache quantization significantly degrades math reasoning performance on AIME25, but applying activation rotation before quantization mostly recovers quality (31.7% → 37.1%). The finding means anyone running large models with Q8 KV cache to save memory has been silently losing quality on reasoning-heavy tasks. The fix requires no model changes — it's a runtime rotation applied in llama.cpp. Debate continues on whether to make it opt-in or default. Reddit: r/LocalLLaMA (225↑, 75 comments).