Reflection patches turned into token-level rewards hit AIME'24 73.3% at 0.08x the training FLOPs of scaled SFT

arXiv 2608.23493·medium signalXBlueskyLinkedInCopy link

SRPO has a model analyze its own finished trajectories, compress the errors into short reflection patches, then score student on-policy rollouts conditioned on those patches to produce dense token-level training signal, with no external critic, separate reward model, or larger teacher. On Qwen3-8B it reports 73.3% on AIME'24 at 0.08x the training FLOPs of scaled supervised fine-tuning, plus 64.7% on WebShop, 76.8% on ALFWorld and 31.2% on SWE-Bench-Lite. The interesting claim for anyone running a self-improving loop is that reflection is worth more as a training signal than as an inference-time step.