DASH Reweights Distillation by How Divergence Evolves Over a Rollout — Free Gains From Distributions OPSD Already Computes

arXiv 2608.06243·low signalXBlueskyLinkedInCopy link

Standard on-policy self-distillation assigns every local teacher-student divergence the same coefficient regardless of position or the discrepancy history preceding it, so it cannot distinguish a spike that follows steady agreement from one in an already-diverging trajectory. DASH (arXiv 2608.06243, Aug 6) maps each local signal's gap from the sequence-level mean to an adaptive propagation gate, then uses those gates to control backward multi-step aggregation of token-level weights. It improves over matched vanilla OPSD reruns on all three mathematical reasoning benchmarks at all three model scales tested, and because it reuses distributions OPSD already computes, it costs no additional teacher or student forward pass. Code at github.com/DBtxy/DASH-OPSD.