Fetching from the wire…
01
02
03
04
05
06
07
08
Models2026-09-15 · source-backed
arXiv 2609.13285 observes GQA still writes both a key and a value at every decode step, and proposes storing only grouped values, reconstructing content keys through a learned linear map that can be absorbed into the query so content keys never materialize. A small shared decoupled RoPE channel carries position through a separately cached positional key. At 350M parameters on 30B FineWeb-Edu tokens, the 16-dimensional positional variant reaches 44.18 average accuracy across five tasks against 44.36 for GQA and 43.88 for MLA. About half the cache for about the same score, with custom decoding kernels still in progress.
Each link below shares sources, entities, or timing with this story.
ArXiv 2603.17946 enables upgrading models from grouped-query attention (GQA) to multi-head latent attention (MLA) via covariance-aware rank-enhanced decomposition. By preserving covariance structure during low-rank decomposition, CARE retains quality while gaining MLA's KV-cac...
Attention-observed selectors like H2O and SnapKV collapse to 0.00-0.33 needle retrieval on a NoPE MLA model, because a long-lived cache must be compressed before the queries that will read it exist. On Kimi Linear, VestigeKV evicts by a query-independent signal already in the...
WorldTrace diagnoses why long rollouts forget: temporal RoPE offsets land outside the trained range so attention can't address stored content, and naive cache compression worsens it by averaging incompatible positional phases. The fix assigns each compressed summary slot a dis...
Translating key-value state from one model into a form another can consume works across scale, architecture, attention configuration, tokenizer and family (arXiv 2608.30963). Llama3.1-70B to Qwen2.5-7B reaches 44.0% accuracy against 45.7% native while dropping latency to 138ms...
Nearly all cache-compaction research assumes a static context where future queries are known offline, which agents never have. Comparing token eviction against attention matching across proxy-query sources on BrowseComp-Plus and WideSearch, compacting a turn immediately often...
SQD argues disaggregated serving built around dense attention is the wrong shape for sparse, linear or sliding-window attention models, and splits decode by quadratic against subquadratic work. For sparse-attention models it separates top-k selection, which must index the full...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.