Agent failure detection at 200 microseconds per step beats LLM-judge monitoring, and deterministic verification beats both

arXiv·high signalXBlueskyLinkedInCopy link

Posted 2026-08-03 (arXiv 2608.02464), this study of 2,823 committed agent episodes across three frameworks, three local models, and gemini-2.5-flash shows a one-class echo-state-network ensemble with CUSUM alarms catches 71% of mid-episode failures at a 5% false-alarm budget (AUROC 0.872) while costing about 200 microseconds per step, three orders of magnitude below a judge call. The more useful finding for builders is that learned monitors do not transfer (AUROC 0.527 cold versus 0.885 recalibrated), whereas a deterministic verifier that recomputes a run's stated total from the tool results actually received catches 60% of failures (96% with a coverage check) with 0 false positives in 63 trials and 0 trips across 1,825 healthy episodes. Closing detection into rollback-and-rerun repair recovered 45% of failures versus a 16% resampling control and lifted task success from 52% to 73% for about one extra model call.