Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-10 · high
The optimal key transform turns out non-orthogonal, and 2-bit quantization recovers accuracy scalar methods lose.
Why now: NOVA-KV posted to arXiv in August 2026, right as long-context agent workloads make cache bandwidth a binding cost on GPU serving.
NOVA-KV reframes key-value cache quantization as transform coding, scoring error against the attention output instead of the raw cached values, per arXiv 2608.04074. That distinction matters. Long-context decode is bandwidth-bound, and cache size sets both token throughput and how many agent sessions fit on a single GPU.
Standard scalar quantization shrinks the cache but degrades long-context retrieval accuracy. NOVA-KV instead derives closed-form optimal transforms straight from calibration statistics, built to satisfy a generalized Parseval relation, per the paper. The resulting key transform is non-orthogonal, a break from how most compression schemes are structured. At two bits per element, it recovers most of the retrieval accuracy that scalar quantization loses.
Here's the bet: if the non-orthogonal result holds outside this paper's own benchmarks, KV cache quantization stops being a storage trick. It becomes the lever that decides how many concurrent agent sessions a GPU can serve. Whoever gets transform-coded caching into a production serving stack first should pack more sessions per GPU than teams still running scalar quantization.
Each link below shares sources, entities, or timing with this story.
NOVA built by Amazon / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (NOVA built by Amazon); both cover GPU, Relevant; overlapping topics (agent, coding).
Shared entity: GPU / Shared topic / Earlier coverage
Both cover GPU; overlapping topics (agent, attention, cache, coding, decode); earlier GPU coverage from 2026-08-09.
Shared entity: GPU / Same source domain / Shared topic / Earlier coverage
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (accuracy, agent, cache).
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (agent, cache).
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (attention, cache).
Shared entity: Relevant / Same source domain / Shared topic / Earlier coverage
Both cover Relevant; reported by the same outlet (arxiv.org); overlapping topics (accuracy, agent).
Shared entity: GPU / Same source domain / Shared topic / Earlier coverage
Both cover GPU; reported by the same outlet (arxiv.org); overlapping topics (agent, cache).
Same source domain / Shared topic / Tension
Reported by the same outlet (arxiv.org); overlapping topics (accuracy, agent, attention, cache); pushes against this story (against).