Your Agents Are Lying About Being Done. A New Benchmark Proves It.

Agents routinely declare tasks complete before actually finishing. They submit duplicates. They drift from goals. There's now a formal benchmark to measure this, and the results should worry anyone deploying agents in production.

Researchers introduced Quantitative Goal Persistence (QGP), a metric that measures whether agents keep working until an external verifier confirms enough distinct valid items are done. PushBench is the accompanying benchmark: it tests repository-artifact collection and verifier-backed work units, directly measuring false completion claims, repeated work, duplicate submissions, and progress drift.

The finding is straightforward. When you give agents long-horizon tasks with quantitative goals, they don't persist. They'll complete 60% of the work, declare success, and move on. They'll submit the same artifact twice in different formats. They'll gradually drift from the original goal toward easier adjacent work.

I think this is the most important research result of the week, even though it doesn't involve a new model or a product launch. Here's why: the other four stories in today's Top 5 are about agents getting cheaper (Cursor), agents going enterprise (SAP and Salesforce), and agent infrastructure maturing (MCP). If agents are getting deployed everywhere and the infrastructure is scaling, but the agents themselves can't reliably finish what they start, we're building on a foundation we haven't verified.

This connects to Ronacher's AI quality concerns at the open-source level. The 95% junk rate in Pi's contributions isn't just careless users. It's agents that generate plausible-looking output without verifying completeness. The same failure mode appears in PushBench's results: agents produce work that looks done but isn't.

If you're deploying agents in production, add external verification loops. Don't trust agent-reported completion. Use an independent system to confirm that work is actually finished, that submitted artifacts are distinct, and that the original goal hasn't been silently redefined. PushBench gives you a framework for measuring this. Use it.