Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-31 · high
LoopArena swaps in a fixed Worker agent so only the Controller's stop-or-retry decisions get graded, and the top score is under 25%.
Why now: The paper posted to arXiv in August 2026.
LoopArena's best-performing loop controller clears just 24.69% strict success across tested coding tasks, according to the LoopArena paper.
That matters for anyone building agent harnesses. It suggests the surrounding loop logic, not the code-writing model, caps performance in these tests.
A single fixed Worker agent writes the code in every test. Separate Controller models see only a structured summary after each round, then decide what to try next, whether to verify, or when to stop.
The paper's cheaper Type II evaluation setting reproduces that same Controller ranking at a Spearman correlation of 0.9747. The correlation is close enough to rank candidate controllers without running full end-to-end tasks.
Paired against that shortcut, tested Controllers cut estimated inference cost by an average of 64.4%. Code for the benchmark is posted on GitHub under AMAP-ML.
The Worker never changes across tests. If a better Controller still can't push strict success past 24.69%, the bottleneck is deciding when to stop, not how to code. Worth watching whether other benchmarks start separating Controller from Worker, since bundling the two hides which part is broken.
Each link below shares sources, entities, or timing with this story.
Shared entity: Code / Same source domain / Shared topic / Earlier coverage / Tension
Both cover Code; reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark, best, code).
Both cover Code; reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark, code).
Shared entity: Code / Same source domain / Shared topic / Earlier coverage
Both cover Code; reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark, code, coding).
Both cover Code; reported by the same outlet (arxiv.org); overlapping topics (agent, code, coding, cost).
Both cover Code; reported by the same outlet (arxiv.org); overlapping topics (agent, benchmark, code, coding).
Shared entity: Code / Same source domain / Shared topic / Earlier coverage / Downstream implication
Both cover Code; reported by the same outlet (arxiv.org); overlapping topics (code, each).
Shared entity: Code / Shared topic / Earlier coverage / Tension
Both cover Code; overlapping topics (agent, code, coding, cost); earlier Code coverage from 2026-06-01.
Shared entity: Type II / Shared topic / Earlier coverage
Both cover Type II; overlapping topics (agent, benchmark, coding, cost, each); earlier Type II coverage from 2026-07-17.