A self-improvement recipe can quietly make your model worse on new tasks.

A new paper shows verifier-driven self-DPO, the common production pattern where a frozen verifier scores candidate generations and the top picks get reinforced, can cause measurable regression when the model later faces new tasks (arXiv 2606.14629). The frozen verifier silently entrenches narrow behavior. If you're running any self-improvement, RLAIF, or self-DPO loop in production, this is a direct caution: your verifier's blind spots become your model's blind spots, and you won't see it until the distribution shifts. Hold out a genuinely novel eval set the verifier never touched, and re-run it every cycle.