SortedRL: Online Length-Aware Scheduling Accelerates RL Training for LLM Reasoning Without Quality Loss

arXiv·medium signal

Microsoft researchers introduce SortedRL, an online length-aware scheduling method that accelerates reinforcement learning training for LLMs by sorting and batching training samples by response length. The approach reduces GPU idle time from padding waste in variable-length RL rollouts, achieving significant training speedups while maintaining the same reasoning quality — directly applicable to scaling agent training on long-horizon tasks.