arXiv: Test-Time Scaling Makes Overtraining Compute-Optimal — T² Laws Rewrite Chinchilla for the Inference Era

arXiv / HuggingFace Daily Papers·medium signal

Researchers from UW-Madison and Stanford (2604.01411) introduce Train-to-Test (T²) scaling laws that jointly optimize model size, training tokens, and inference samples under fixed end-to-end budgets. Across eight downstream tasks, optimal pretraining shifts radically into the overtraining regime when accounting for inference cost — smaller models trained longer become compute-optimal when you plan to sample multiple answers at test time. Validated by pretraining models in the predicted optimal region, confirming substantially stronger performance vs. training-only scaling.