Agents know when they'll fail, not how to fix it

Agents predict their own failure with 88.5% accuracy but fail to pick the right fix, per a new study of 4,181 competition math problems.

For teams building multi-agent systems, that gap costs tokens. A frozen protocol picked in advance can burn 71,300 tokens for 73.8% accuracy, worse than a policy that escalates only when confidence is low.

The study tested four setups: direct solving, iterative self-correction, planner-executor-reviewer collaboration, and multi-agent deliberation. Detecting an incoming failure scored 0.8847 AUROC, per the paper. Picking which protocol would fix it did not transfer at the same level.

Confidence-based escalation hit 78% accuracy at 45,000 tokens. A retrospective oracle, picking the best protocol after the fact, reached 92.4% accuracy, per the paper. That 18.5-to-28.9-point gap is what's actually on the table in a real deployment.

This sits next to other recent work treating agent orchestration as an engineering problem. One study composes guardrails as an algebra instead of a rule list; another finds repo layout changes prompt-injection resistance.

The paper doesn't say whether this confidence signal holds up outside competition math, where answers are checkable and problems are bounded. Routing on self-reported confidence in open-ended coding or research work is untested here.