AutoSaddler Treats the Agent Harness as Code and Patches It From Failure Traces, Gaining 9-10 Points on Three Benchmarks

arXiv 2608.23041·high signalXBlueskyLinkedInCopy link

AutoSaddler formulates harness improvement as offline learning, combining failure-trace diagnosis, structured patch generation that edits the harness as source code, and validation-based update selection over mini-batches of failures. It gained 9.0 points on GAIA2, 9.6 on SWE-Bench Pro, and 10.0 on Terminal-Bench 2.0 over the corresponding base harnesses. Ablations name the three ingredients that matter: deep debugging rather than shallow reflection, targeted modifications rather than unconstrained editing, and generalization-aware selection rather than trajectory-specific repair.