Fetching from the wire…
Infra2026-09-04 · source-backed
Attention-observed selectors like H2O and SnapKV collapse to 0.00-0.33 needle retrieval on a NoPE MLA model, because a long-lived cache must be compressed before the queries that will read it exist. On Kimi Linear, VestigeKV evicts by a query-independent signal already in the cache, the 64-dimensional decoupled branch that NoPE training repurposes from a RoPE vestige into a salience channel, reading 11% of each row and moving non-top rows into a GPU-resident archive. No training, no quantization, no kernel change. Retrieval holds 1.00 at 8x and 0.92 at 32x from 8k to 65k. The identical operator on a RoPE MLA collapses to 0.08, so the effect is NoPE-exclusive. arXiv 2609.03949
Each link below shares sources, entities, or timing with this story.
Sebastian Raschka's July 28 teardown argues K3 is less exotic than the release framing suggests: a scaled production version of Kimi Linear with Kimi Delta Attention as the hybrid attention layer and LatentMoE compressing large linear layers by down-projection. The genuinely n...
Moonshot released K3's open weights July 26 with official guidance calling for 64+ accelerators. WASTE (1,366 stars, created July 28) runs it on a 64GB MacBook Pro at 0.45-0.62 tok/s, keeping the 27.28GB trunk resident and streaming experts from NVMe with 3-bit residual vector...
WorldTrace diagnoses why long rollouts forget: temporal RoPE offsets land outside the trained range so attention can't address stored content, and naive cache compression worsens it by averaging incompatible positional phases. The fix assigns each compressed summary slot a dis...
Wafer.ai ran K3 at TP8 on a single MI355X node: 952 tok/s aggregate, 118 tok/s single-stream decode, ~13k tok/s steady-state prefill after fixing missing PyTorch sampling functions and AITER MLA prefill kernels via SGLang and ROCm. Against a two-node TP16 B200 deployment that'...
arXiv 2607.24667 recasts eviction as estimation of a hidden reuse signal along a commit-lag axis, with StreamingLLM/H2O/SnapKV at lag 0 and Belady's optimum at full future knowledge, then fills the middle: wait a bounded number of steps, observe what a correct near-future pred...
Alagappan Valliappan shows built-in Multi-Token-Prediction draft heads run full attention over the entire KV cache at every draft step, so at million-token context the "negligibly cheap" draft dominates cost and deep native drafts can go net-negative. Applying a StreamingLLM-s...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.