The best coding agent resolves 25% of long-horizon tasks: Microsoft's LoopsBench shifts evaluation from harness engineering to loop engineering
LoopsBench comprises 112 tasks from authentic sources across 8 programming languages and 9 domains, structured as a dependency DAG over separately testable development units with 5,300+ executable test components — measuring sustained iterative development rather than isolated task completion. Claude Opus 4.7 with Claude Code and continuation mechanisms was the strongest performer at only 25% of tasks resolved, and the failure analysis is the useful part: recorded plans recover only part of the source-recovered prerequisite DAG, and regression events remain visible across every loop profile tested. Builders should read this as evidence that agent-authored plans systematically under-capture prerequisites, and that regression gating between loop iterations is not optional; the benchmark is open-sourced at microsoft/Loopsbench.
↳ Follow the thread