Cost-cutting AI cascades read 3% error while it hits 32%

A common setup routes most queries to a cheap "student" model, then escalates anything that looks hard to a frontier model that checks the work. It's supposed to hold down inference cost while catching errors where they're most likely.

A new paper measured how well that verifier catches errors, and found its blind spot moves with the student's own skill. At a 0.5B-parameter student, the verifier misses 12% of the student's escalated errors. At 32B, it misses 55%. The verifier gets worse exactly as the cheap model gets more capable, which is the regime cascades exist for.

Swapping in a frontier verifier fixes the blind spot, cutting it to about 5%. But that verifier then escalates 46% of hard MATH queries, against a 39% true error rate. It sends more work to the expensive model than the errors justify, eating into the savings the cascade is supposed to deliver.

The paper also tried fine-tuning the student on the verifier's rejections, a common way to make a cascade self-improve. Performance degraded and eventually collapsed, across every teacher model tested.

The starkest gap sits in the metric itself. The error rate computed through the verifier stayed flat near 3% the whole time, while the error rate delivered to users climbed to 32%. A team watching only the cascade's own dashboard would have seen nothing wrong.