The Same GRPO Recipe on Three From-Scratch LLMs Produced Three Different Outcomes, and the Middle Model Broke Worst

r/MachineLearning·low signalXBlueskyLinkedInCopy link

A researcher pre-trained three models from scratch in raw PyTorch (353M MHA, 316M differential+GQA, 672M XSA+GQA on 10B/10B/30B tokens), then ran identical SFT and GRPO with the same curriculum, reward function, hyperparameters and KL coefficient. WikiText perplexity after SFT to GRPO moved +0.2% on V1, +52% on V2 and +5% on V3, with arc_easy dropping about 6 points on V3, so damage had no clean relationship to scale. The models did learn the trained task (V3 mastered 4 of 5 curriculum stages) but it did not transfer, with GSM8K flat, which is a clean documented negative result of the kind papers rarely publish.