Almost every deployed model misreports its own task progress mid-execution, so agent frameworks should not gate control flow on it

arXiv 2609.08589·high signalXBlueskyLinkedInCopy link

Evaluating task-progress reporting on τ²-bench and StageIF, a testbed that places reporting checkpoints across a task's lifecycle, reliability turned out to depend on which stage the task had reached. Most deployed models lose accuracy once work is under way and recover once the task is done; the newest generation closes that mid-task dip and instead becomes conservative at the finish line, under-reporting completion. If your orchestrator asks the model 'are you done?' to decide whether to continue, you are reading a signal whose error pattern shifts with the stage, and the paper's explicit conclusion is to stop controlling flow on the model's state reports alone.