RECLAIM: the best agent reproduces 41% of NeurIPS 2025 papers when code, data and weights are all released.

arXiv 2609.28850 fixes per paper the result to reproduce, the success criterion and a GPU-hour budget across 100 papers, and an LLM grades from logs rather than from the agents' own reports. Best of four agents: 41% on Run-tier, 27% Retrain-tier, 15% Reimplement-tier. Failed runs used only 29% of their budget on average, so they gave up rather than ran out. The single most common error, 63 of 400 runs, was writing the method without checking any intermediate number against the paper.