Hide tool latency by making the agent speculate its own next tool call — Hit@1 44.1% → 61.2% with no separate draft model

arXiv 2607.25816·medium signalXBlueskyLinkedInCopy link

Instead of a separate speculator model or cached traces poorly aligned with the deployed agent, this trains one model to run in two modes: agent (solving the task) and speculator (predicting the next tool call from a partial trajectory), reusing the prefix KV cache so speculation is nearly free. Joint agent-speculator RL derives speculation targets from the agent's own rollouts and alternates agent and speculator updates. Next-tool-call Hit@1 rises from 44.1% to 61.2% on Qwen3-4B and 48.9% to 66.3% on Qwen3.5-4B while preserving task success — correct predictions let you pre-execute and hide the wall-clock wait.