Google's TurboQuant Gets Community llama.cpp Implementation — 6x KV-Cache Compression, Zero Accuracy Loss at 3-Bit Quantization

r/LocalLLaMA / Google Research / GitHub·medium signal

Within 24 hours of Google publishing TurboQuant (ICLR 2026), the llama.cpp community built a working CPU implementation achieving TQ3 (3-bit) quantization with MSE=0.034 matching the paper's results and 4.9x compression vs FP16. A practitioner benchmarked it against the paper's claims (270 upvotes, 72 comments), validating 6x KV-cache memory reduction and 8x attention speedup on H100s with zero accuracy loss up to 104K context. The implementation is pure C with no dependencies.