Best AI model catches risky agent moments 40.74% of time

PASTABench tested 16 AI models on catching risky agent conversations before they escalate. The best one hit the right moment only 40.74% of the time.

That gap matters for anyone building an agent overseer. Miss the window and the risky action already happened. Flag too early and users learn to ignore the warning.

The PASTABench paper built 1,139 multi-turn agent trajectories across 5 risk categories. Each trajectory got two human-annotated markers.

An Earliest-Signal turn marks where a risk first becomes visible. A Trigger turn marks the point past which intervening is too late. The space between them is the Optimal Intervention Window, and models earned credit only for flagging the conversation inside it.

Small models turned out to be the surprise. Several posted safety scores competitive with much larger models, but the paper attributes this to keyword hypersensitivity rather than real risk understanding. Those models flagged individual trigger words. They didn't track where the conversation was heading.

The paper doesn't say whether the gap closes with more training or a different architecture, and it doesn't name which of the 16 models scored highest. What it does establish is a bar. Even the best model missed the intervention window more than half the time.