HySparse2 Lets Prefill Exit Halfway Through an 80B-A3B MoE by Building All Cross-Decoder KV From Self-Decoder States

arXiv·medium signalXBlueskyLinkedInCopy link

HySparse2 targets agent workloads that produce short actions over long tool observations. It pairs a YOCO-style self-decoder and cross-decoder, bridged only at full-attention layers, with token-level sparse attention. Because every cross-decoder KV cache comes from self-decoder hidden states, prefill can skip the cross-decoder layers entirely. On an 80B-A3B MoE model it beat HySparse and hybrid sliding-window attention on long-context retrieval and multi-turn agentic tasks while cutting prefill compute and KV storage.