Declarative Attention Lets a Model Announce Where to Look and Skips Half the KV Cache Read

arXiv 2609.02737·high signalXBlueskyLinkedInCopy link

Instead of scoring context relevance with an external proxy that still costs O(N) per decoding step, Declarative Attention has the model declare its attention scope inside its own chain-of-thought using three modes — global (full context), focus (a specific region), and local (recent output only) — which the inference engine parses like tool calls and uses to skip most of the KV cache read. Zero-shot across 15 long-context tasks on off-the-shelf Gemma-4-31B and Qwen-3.6-27B, it cut total attended tokens during decoding by 52.0% and 31.1% with accuracy drops of only 1.27 and 2.75 points. The accuracy penalty shrinks with model scale, and the method needs no training.