ERPBench: agents save the ERP form in 85% of runs and write the correct value in as few as 3%

arXiv·medium signalXBlueskyLinkedInCopy link

arXiv 2609.17885 (15 Sep 2026) evaluates six closed and open-source screenshot-only computer-use agents against a live, reproducible ERP system, scoring each task against ground-truth database values rather than screen state. Strong general GUI performance does not transfer: agents reach the right form and save it, yet the stored record is frequently wrong, with one configuration saving in up to 85% of runs while writing the correct value in as few as 3%. The release includes a production harness that gates agent actions behind human approval, which ERPBench itself runs autonomously.