A Compute-Centric Taxonomy of Parallelism for RL Post-Training, Framed With the Work-Depth Model

arXiv 2608.27046·low signalXBlueskyLinkedInCopy link

Reasoning-model training via RLVR needs millions of GPU-hours and tightly coupled multi-model pipelines, making it as much a distributed systems problem as an algorithmic one, yet the parallelism literature has treated it as the latter. This work systematizes PPO, GRPO and their variants under a compute-centric analysis, then builds a taxonomy spanning traditional data, tensor, pipeline, sequence, context and expert parallelism alongside RL-specific forms such as disaggregated placement, stage fusion, hybrid parallelism and asynchronous execution. The work-depth model of parallel computing is used to keep the taxonomy rigorous and portable across frameworks, and the paper closes with practical guidelines for building cost-effective RLM training stacks.