CoRT Gets Token-Level Credit From Rubric-Guided RL Without Training a Second Model, Adding 4.4 Points Over Response-Level GRPO

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

Submitted July 28 by Bo-Wen Zhang and seven co-authors and sitting at 60 upvotes on Hugging Face Daily Papers, CoRT attacks the coarseness of rubric-conditioned GRPO, which assigns one reward to a whole response. It rescores each response twice — with and without the rubric in context — and uses the counterfactual likelihood contrast as a proxy for which tokens actually depend on the rubric, yielding token-level credit with no auxiliary scoring model to train. Reported gain is 4.4 percentage points on average over response-level GRPO across multiple benchmarks and model configurations. For anyone running LLM-judge or rubric-scored evaluation loops, the trick generalizes beyond RL: the with-and-without-context likelihood delta is a cheap attribution signal.