Amazon researchers compressed winning ML agent strategies to 16 tokens and argue that is why the agents don't overfit

Amazon Science·medium signalXBlueskyLinkedInCopy link

Martin Bertran Lopez and Aaron Roth published 'What fits (into few tokens) doesn't overfit' on 2026-09-10, asking why ML research agents that iterate against a validation set do not overfit it. Across eight datasets spanning tabular and image classification, language modeling, diffusion and reward modeling, winning strategies compressed to 16-32 tokens with no performance loss, including one language-modeling recipe surviving as 'QKn 12L768 Mu .1 R² b2M 4x'. Their argument is that short descriptions cannot cheat because there is no room to memorize, and that compression therefore works as a cheap overfitting test: genuinely overfit strategies lose their validation-specific gains when squeezed.