RetireOPD lets AI agents fire their own teacher

RetireOPD trains AI agents by copying a teacher model, then drops that teacher once the student stops closing the performance gap between them. That comes from the paper on arXiv, which tests the method on Qwen2.5 models from 1.5B to 7B parameters. Across those sizes, the student beats the teacher it trained from every time, using reinforcement learning to close the gap that copying alone couldn't.

The mechanism is a discrepancy check. RetireOPD tracks the gap between student and teacher performance. Once that gap stops shrinking, and the student reaches a target fraction of the teacher's success rate, it drops the teacher. Training finishes with reinforcement learning alone.

The results hold across ALFWorld and WebShop, two benchmarks where agents complete multi-step tasks like navigating a simulated home or shopping site. Against a plain RL baseline, RetireOPD improves ALFWorld success 14.1 to 18.8 percent and WebShop accuracy 11.8 to 19.0 percent.

Most distillation setups treat the teacher as a ceiling. RetireOPD's stopping rule is what unlocks the reinforcement learning phase that pushes the student past it. It changes how teams might think about picking a teacher model. The teacher stops being the target and becomes a stage to outgrow.

The paper doesn't say how the discrepancy threshold was tuned, or whether it needs retuning for other tasks. Anyone applying this to a different agent benchmark will have to work that out themselves.