One-Shot On-Policy Distillation Works, and the Driver Is Long Chain-of-Thought Length, Not Token Entropy

arXiv 2609.05198·medium signalXBlueskyLinkedInCopy link

arXiv 2609.05198 finds 1-shot on-policy distillation is consistently effective across every sampled training example, with harder examples yielding larger gains. The analysis attributes the improvement not to high token entropy but to the longer CoT paths hard problems naturally generate, which keep the student aligned with the teacher over long reasoning horizons and teach reflection patterns absent from short CoTs. A hard-example-only selection method — including 'unsolvable' examples that exceed the teacher's own capability — trains successfully with just 8 selected examples across four models from 1.5B to 7B.