DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes Without Teacher Models

arXiv·high signal

DenoiseRL trains reasoning models via verifiable-reward RL by conditioning on truncated incorrect prefixes from weak models, learning to denoise corrupted reasoning states and recover correct solution paths. Eliminates the need for expensive teacher models or curated data while improving reasoning performance and training efficiency. Code released on GitHub (ALEX-nlp/DenoiseRL).