On-policy distillation then RL beats every joint combination of the two.

Prior work fuses OPD's dense token-level supervision with RLVR's sparse reward in a single step, either weighted-additive or teacher-modulated advantage rescaling. A plain two-stage OPD-then-RL scheme beats pure OPD, pure RLVR and all joint baselines across logic and math benchmarks. The mechanism: OPD expands the student's coverage of teacher-supported solutions and RL sharpens within that support, while joint optimization makes the signals interfere. The OPD validation score tells you when to switch, and OPD is a better cold start for RL than SFT. arXiv 2609.04108