Co-RL Trains Reasoning With Peer Rewards Instead of Labels, and Argues Cohort Diversity Is What Stops the Self-Reinforcing Collapse

arXiv (via HuggingFace Daily Papers)·medium signalXBlueskyLinkedInCopy link

Co-RL (arXiv 2608.17253, v1 August 18, v2 August 19) trains multiple independent models against each other using peer-derived rewards with no ground-truth labels, reporting average gains of 3.0-8.6% across seven text benchmarks for LLMs and 2.3-7.2% across four multimodal benchmarks for VLMs, matching or beating supervised methods. The mechanism claim is the interesting one: varying architectures, model sizes and rephrased training samples across the cohort is what breaks the correlated errors that otherwise drive self-reinforcing feedback loops and training collapse. For anyone running multi-agent verification, this is direct evidence that identical verifiers are worse than deliberately heterogeneous ones. Code is on GitHub; 30 pages, 5 figures, 11 tables.