DocOps Benchmark Names Three Agent Failure Modes in Document Work — Including Destructive Edits to Structural Metadata

arXiv 2607.19865·medium signal

DocOps is a deterministically verifiable benchmark for agents manipulating everyday documents, built on a taxonomy that breaks operations into atomic dimensions and escalating workflow complexity. Across closed- and open-source models in multiple agentic harnesses, even frontier configurations degrade sharply on highly coupled long-range tasks, and behavioral analysis isolates three recurring failure modes: long-term state tracking collapse, shallow semantic verification, and destructive editing of structural metadata. That third one is the one to design around — agents quietly corrupt document structure while reporting success.