A model can now tell its inference engine where to look

Most KV cache tricks bolt on an external scorer to guess which tokens in a long context still matter, and that scorer runs its own pass over the sequence every decoding step. A new method skips the guesswork by having the model say what it needs, in its own chain-of-thought.

Declarative Attention gives the model three modes to declare: global for the full context, focus for a named region, and local for just its recent output. The inference engine reads these declarations the way it reads tool calls, then skips reading most of the KV cache for that step. Tested zero-shot on off-the-shelf Gemma-4-31B and Qwen-3.6-27B across 15 long-context tasks, per the paper, it cut total attended tokens during decoding by 52.0% on Gemma and 31.1% on Qwen. Accuracy fell 1.27 points on Gemma and 2.75 on Qwen.

No fine-tuning, no extra scoring model, no architecture change. The models already had this capability sitting in their instruction-following behavior; the paper just gave the inference engine a grammar to parse for it.

The accuracy penalty shrinks as model scale goes up, which is the detail worth tracking. If that trend holds at frontier scale, self-declared attention scope could end up cheaper and more accurate than the retrieval-augmented and external-scorer approaches teams build for long-context serving. The paper doesn't say whether the technique holds up on tasks that require jumping between multiple distant regions in a single step rather than settling into one scope, which is exactly the case a focus-only declaration would handle badly.