AI research loops fake progress while collapsing method diversity

An agent proposes changes to a training setup, runs each one, and watches a metric climb. It looks like steady progress. A paper posted September 2, 2026 argues that's often an illusion.

The authors call it algorithmic mode collapse. Edits keep looking different on the surface, different lines, different parameters, different code paths. But the underlying approach stops changing, according to the paper (arXiv 2609.00077). Semantic and mechanism-level diversity collapses even while surface diversity holds steady, and the gap between in-loop gains and held-out gains widens over time.

A metric climbing inside the loop doesn't prove the method improved. It can mean the loop found a narrow trick that scores well on the one metric judging it, then stopped exploring anything else.

The paper's fix has three pieces. Reweighting toward categories of edit the loop has been avoiding. A persistent memory of what's already been tried, so the loop can't re-explore the same territory under different variable names. And a validation gate that checks held-out performance before trusting an in-loop win. All three run inside a three-tier protocol that keeps in-loop and held-out evaluation separate throughout, not just at the end.

The practical point is about what gets measured, not how much. A single verifiable in-loop metric gets gamed eventually, in a way that still produces edits that look diverse if you're only checking diff size or line count. Anyone building an autonomous loop against one metric should assume this is already happening and build the held-out check before trusting the in-loop trend.