U-OPSD beats GRPO on math tests with no ground truth

U-OPSD trains reasoning models to solve math with no answer key, teacher model, or reward feedback. The method comes from a paper posted to arXiv in August 2026.

That removes one of the standard costs of training reasoning models: labeled answers or a separate reward model to check against. It's aimed at math benchmarks with one checkable right answer, per the paper.

GRPO is the reinforcement learning method this kind of self-distillation research gets measured against. In thinking mode, U-OPSD beat it by 0.7 to 1.1 points across the same benchmark set, without GRPO's environment feedback.

The method samples multiple rollouts per problem, then builds a pseudo-solution by majority vote under a self-consistency threshold. It conditions a teacher distribution on the shortest pseudo-solution, then distills that into prefixes of the model's longest incorrect completion. The label is the model's own consensus answer, not a verified one.

Across AIME24, AIME25, HMMT25, MATH500 and AMC23, U-OPSD lifted Qwen3 non-thinking mode accuracy 8.5 points at 4B and 10.7 points at 8B. That beat supervised OPSD, per the paper.

Every one of those benchmarks has a single, checkable right answer. The paper doesn't say whether majority-vote consensus holds up on reasoning tasks without one clean answer to converge on. That's most of what reasoning models get used for outside math contests.