Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-17 · high
1,221 people tested three coding agents on ICML papers, and human-guided runs beat agents working alone.
Why now: The hackathon ran July 15 to August 2, giving peer review a data point instead of a guess about what agent-assisted checks catch.
Hugging Face ran 2,226 of ICML 2026's accepted papers through agent review from July 15 to August 2. 1,221 people used Claude Code, Codex and Cursor for the check, per Hugging Face. That's more than a third of the conference's 6,352 accepted papers, checking that used to cost a reviewer a weekend per paper.
The hackathon logged 6,816 logbooks and 2,962 cloud jobs. 51% of the examined papers had at least one claim the agents verified. 23% had at least one claim flagged false, with 49 papers falsified outright. Independent teams checking the same paper landed on conflicting verdicts in 242 cases.
Agent-only runs hit limits on scale-dependent behavior, according to the results. Human-guided workflows, where a person steered the agent's checks, were the most reliable setup in the hackathon.
"Checking a paper carefully used to cost a reviewer a weekend; an agent can attempt it in an afternoon," the hackathon's organizers wrote. Agent-only runs still missed cases that human-guided runs caught, which is why the 242 conflicting verdicts matter as much as the 49 outright falsifications. Conferences reading this as a case for agent-only review are reading past the results.
Each link below shares sources, entities, or timing with this story.
Claude Code competes with Cursor / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Claude Code competes with Cursor); both cover Agent, August, Claude Code, Codex; overlapping topics (agent, claude, cloud, code).
Claude Code competes with Cursor / Shared entities / Same source domain / Shared topic / Tension
Linked by a graph relationship (Claude Code competes with Cursor); both cover August, Claude Code, Hugging Face, July; reported by the same outlet (huggingface.co).
Claude Code competes with Cursor / Shared entities / Shared topic
Linked by a graph relationship (Claude Code competes with Cursor); both cover Agent, August, Claude Code, Codex; overlapping topics (agent, august, claude, code).
Claude Code competes with Cursor / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Claude Code competes with Cursor); both cover Agent, August, Claude Code, Codex; overlapping topics (agent, claude, code).
Linked by a graph relationship (Claude Code competes with Cursor); both cover Agent, Claude Code, Codex, Cursor; overlapping topics (agent, claude, code).
Cursor benchmarked against Codex / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Cursor benchmarked against Codex); both cover Agent, Claude Code, Codex, Cursor; overlapping topics (agent, claude, code).
Claude Code competes with Cursor / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Claude Code competes with Cursor); both cover August, Claude Code, Hugging Face, July; reported by the same outlet (huggingface.co).
Cursor supports Claude / Shared entities / Shared topic / Earlier coverage / Tension
Linked by a graph relationship (Cursor supports Claude); both cover Agent, Codex, Cursor, July; overlapping topics (agent, claude, cloud).