Off-Context GRPO Recovers Learning Signal on Problems Where Standard RLVR Gets Zero Reward

arXiv 2607.19313·low signal

RLVR breaks down on hard problems: if a model can never produce a correct solution, every rollout is wrong and the gradient carries no signal. OC-GRPO injects privileged guidance during training — solution prefixes and similar hints — to generate successful rollouts, then uses importance correction to steer those guided trajectories back toward the original unguided objective, avoiding the instability that naive hint-injection causes. It reports a 3.9% absolute (13.8% relative) average improvement over vanilla GRPO on mathematical reasoning benchmarks as a minimally modified variant with negligible additional cost.