Prime Intellect Makes Multi-Agent RL First-Class in PRIME-RL, and Had to Invent Hierarchical GRPO Because Classic GRPO Can't Express Role-Aware Credit
Published August 7, the PRIME-RL update introduces `Agent` and `Env` as first-class abstractions where `Env.run(task, agents)` programs arbitrary multi-agent control flow and every finished agent run auto-joins the resulting Episode, letting you pick which roles actually learn. Four implemented environments ship: Agentic Judging (judge grades solver traces), User-Sim, Proposer-Solver, and Kuhn Poker self-play. The sharpest technical detail is a credit-assignment problem others will hit too — solver attempts must be compared against attempts on the same proposed problem while proposer traces compare against proposals from the same seed, a hierarchy plain GRPO cannot represent, so Prime Intellect shipped Hierarchical GRPO; the Proposer-Solver reward is calibrated so learnability peaks at a 50% solve rate.
↳ Follow the thread