Researchers trace distillation's length inflation to EOS mismatch

On-policy distillation inflates output length because teacher and student models bet on different EOS tokens, per the paper. Across Qwen3, Llama and Gemma, the base student and its post-trained teacher favor different stopping tokens even when their declared stopping sets match. The student's preferred way to end a response gets suppressed during training, and the teacher's alternative doesn't reliably take its place.

Matching the two models' stop-token lists doesn't close the gap. What works is treating functionally equivalent EOS tokens as one shared semantic stopping action, a correction that holds across all three model families.

That fix doesn't clear everything up. A second, separate inflation shows up late in training and survives the correction.

For anyone building distillation pipelines, the finding says the stopping behavior needs checking beneath the token list, rather than assuming matching IDs is enough. The unresolved late-training inflation means teams still fighting length inflation can't assume one fix covers both sources.