Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-07-02 · high
It's a harder, more realistic test than the single-function fixes most benchmarks use for judging legacy-code agents.
Why now: As of July 2, most cited agent benchmarks still score single-function fixes, not the multi-file breaks RepoRescue tests.
RepoRescue tests coding agents by breaking a codebase with dependency and version changes spread across many files at once, per the paper posted on arXiv. That's harder than the single-function fixes most benchmarks score. It's also the test that predicts whether an agent can be trusted on an old codebase, not a fresh one.
Most published agent benchmarks still score single-function fixes: patch one broken call, pass the test, done. RepoRescue skips that setup. It drops agents into a repo where a dependency bump or version change has broken things across many files at once. That's closer to what happens when a legacy project falls behind on updates.
Anyone who's tried to push a coding agent through a multi-file version bump already suspects single-function benchmarks hide the hard part.
Single-function benchmarks are the wrong yardstick for judging whether an agent can handle real legacy work. The model that handles RepoRescue's whole-repo breaks best is the one worth trusting with an actual dependency migration.
As of July 2, most cited agent benchmarks still score single-function fixes, not the multi-file breaks RepoRescue tests.
Each link below shares sources, entities, or timing with this story.
Shared entity: Anyone / Same source domain / Shared topic / What happened next
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, anyone, benchmark, broken).
Shared entity: Anyone / Same source domain / Shared topic / Earlier coverage / Tension
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, anyone, benchmark).
Shared entity: Anyone / Same source domain / Shared topic / What happened next
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, anyone, benchmark).
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, anyone, coding).
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, anyone).
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (anyone, benchmark).
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, coding).
Both cover Anyone; reported by the same outlet (arxiv.org); overlapping topics (agent, anyone).