A Small Draft Model Scoring an Agent's Own Output Predicts Failure Before Execution, Cutting Error Rate 6-8 Points

arXiv 2609.05274·high signalXBlueskyLinkedInCopy link

arXiv 2609.05274 inverts speculative decoding: a small open-weight draft model scores a black-box agent's already-generated trajectory in a single forward pass, needing no logits, weights, activations, or repeated sampling. Phase-aware features separating reasoning from action spans are calibrated against a verifiable objective to produce a failure-likelihood score. Wired into a pre-execution veto gate on Qwen3-Coder-480B and Claude 3.5 Sonnet, it cut execution error rate by 6-8 percentage points and token cost by 14-19%, transferring to out-of-distribution benchmarks without retraining.