Raschka pushes back on Astra hidden-reasoning theory

Sebastian Raschka is pushing back on the theory that OpenAI's Astra model hides its reasoning inside a looped transformer architecture. In his Ahead of AI post, he argues the gains most likely trace to training recipe and data, not a new way of running the model.

The distinction matters because it changes what other labs should copy. If looping is the source of Astra's gains, competitors need a new architecture. If it's the training recipe, they need better data and post-training, a much harder thing to reverse-engineer from outside.

Raschka's strongest evidence is a comment from OpenAI's chief scientist that compute depth stays within a factor of two of GPT-4. That's a narrow range for anyone claiming a fundamentally different architecture is doing the heavy lifting.

He traces weight-reused looped stacks back to Universal Transformers in 2018, then through recent small models. Nanbeige4.2-3B runs 22 blocks twice, Ouro-Thinking 2.6B runs 48 blocks four times, and Mixture-of-Recursions varies the loop count per token. None of these are new tricks. They're a known technique that keeps resurfacing at small scale.

The theory he's rebutting says looping lets a model reason internally without writing out the tokens, producing shorter answers that hide the real work. Raschka's counter is that shorter output reflects capability and post-training tuning, not concealment. He points to GPT-5.6 Sol using 80% more tokens than Luna to hit similar performance. If length tracked hidden computation, that gap should run the other way.

Raschka doesn't say what changed in Astra's training data or recipe to produce the reported gains.