ExANS Gets 1.51x Lossless BF16 KV Cache Compression at 622 GB/s Decode on an H100
OpenLake / Hacker News·medium signal
Exponent Aligned Asymmetric Numeral Systems, published 2026-08-04, splits BF16 into its 8-bit exponent field and its sign/mantissa, aligns exponents to a block-local center, then entropy-codes only the low-entropy exponent stream while storing sign and mantissa raw. Reported H100 numbers: 1.51x lossless ratio, 622 GB/s median decode (peak 628) and 391 GB/s encode. Because decode far exceeds typical ~50 GB/s network bandwidth, remote KV cache transfer becomes bandwidth-bound rather than codec-bound, cutting time-to-first-token. Shipping in OpenLake v0.8 (Apache-2.0, Rust, 2,307 stars) with a vLLM integration.