Agents beat independent sampling on tokens-to-progress at first, then fall below it.

Elo-per-token analysis tracks the best solution at each token budget and uses a Bradley-Terry model to aggregate within-task orderings into cross-task ratings, applied to four general agents on four open-ended benchmarks with sessions up to 100M tokens (arXiv 2609.15309). Independent sampling is the reference where Elo grows linearly with log compute. Agents start above it, their marginal gains diminish, and eventually they fall below. The strongest human contestants improve superlinearly over contest time. Practical read: when an agent stalls, extending the budget is the wrong move.