TurboQuant Community Implementations Hit GitHub — PyTorch, MLX, and llama.cpp Ports of Google's 6x KV Cache Compression (ICLR 2026)

GitHub·medium signalXBlueskyLinkedInCopy link

Independent developers have built working implementations of Google's TurboQuant paper (ICLR 2026) in PyTorch, MLX (Apple Silicon), and C/CUDA for llama.cpp, despite Google not releasing official code. The algorithm compresses LLM KV caches by 6x at 3-bit precision with 99.5% attention fidelity — no training or fine-tuning required. Works as a post-processing step on Gemma, Mistral, and other open models. The llama.cpp discussion thread is driving practical adoption.