Argument That RLVR Post-Training Makes 'Next-Token Predictor' the Wrong Description of a Modern LLM

Garrin McGoldrick / Hacker News·low signalXBlueskyLinkedInCopy link

Garrin McGoldrick's September 4 post (edited September 5) argues the phrase describes only the inference mechanism and obscures what the model learns. His distinction is that pre-training learns by predicting tokens present in data, while reinforcement learning with verifiable rewards changes the objective entirely, maximizing a reward found through exploration rather than matching observed tokens. He makes the case with pseudocode contrasting the two training loops and a chess analogy between a grandmaster-game imitator and an exhaustive game explorer, with no empirical results; the 301-comment HN thread is where most of the disagreement lives.