RISE Builds a Synthetic Teacher by Extrapolating the Model's Own RLVR Trajectory, No External Model Needed

arXiv 2609.05295·medium signalXBlueskyLinkedInCopy link

arXiv 2609.05295 addresses on-policy distillation's teacher bottleneck — external teachers have distribution mismatch, self-distillation is capped by in-context learning capacity. RISE extrapolates the displacement between the current checkpoint and a trailing anchor, in parameter space or output logit space, converting a sparse outcome-induced parameter update into a dense token-level target with no external model or privileged conditioning. Because the teacher refreshes every iteration as the student improves, distillation becomes recursive rather than one-shot; it beats RLVR-only and on-policy self-distillation on math, multi-domain STEM, code generation, and multi-turn agentic tasks.