On-Policy Distillation Has a Degenerate-Agreement Failure Mode; TIDE Lifts Avg@8 From 6.9% to 20.3% and Cuts Response Length 3.6x
Posted 2026-08-10, this paper shows students in on-policy distillation can exploit repetitive loops to hit near-perfect token agreement with the teacher while producing globally broken responses. Reframing the objective around mismatch instead of agreement, it splits errors into student-excess tokens (teacher assigns near-zero probability, log-ratio corrections blow up) and student-deficit tokens (teacher prefers them, student rarely samples them). TIDE applies bounded Hellinger shaping plus analytic teacher top-K injection; across Qwen3 teacher-student pairs on math reasoning it beats standard OPD, and under strong mismatch improves Avg@8 from 6.9% to 20.3% while shrinking average response length 3.6x. Code at github.com/yzc-666/TIDE.
↳ Follow the thread