IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse

arXiv 2603.12201·high signal

Observes that DeepSeek Sparse Attention (DSA) reduces core attention from O(L²) to O(Lk) but its lightning indexer retains O(L²) complexity—IndexCache exploits high cross-layer index stability to amortize this cost by reusing token selection indices across layers. Results in significant wall-clock throughput improvements for long-context agentic workloads without accuracy loss. Drop-in optimization for any system using DeepSeek-style sparse attention in production long-context inference.