Privileged Self-Practice Beats On-Policy Self-Distillation for Multi-Turn Agents, Up to 61% More Resolved on SWE-bench Verified

arXiv·medium signalXBlueskyLinkedInCopy link

arXiv 2609.29051 (Su, Kumar, Ping et al.) shows that on-policy self-distillation with privileged information teaches multi-turn agents to act as if they had seen information they never observed, sometimes scoring below the untrained base model. Their fix, Privileged Self-Practice, moves the privileged hint from the loss into the sampler: when rollouts mostly fail, an analyzer writes a short per-task instruction, the task is resampled with it in context, and training uses unchanged GRPO. It was the only method to beat plain GRPO consistently across AppWorld and SWE-bench Verified with three student models, improving task-goal completion up to 65% and resolved rate up to 61%.