Google Research released RRSI: regularizing harness self-improvement so the gains survive off the training split

arXiv 2609.24972·high signalXBlueskyLinkedInCopy link

RRSI attacks the failure mode where an agent that rewrites its own prompts, control flow and memory memorizes its training tasks: big in-distribution gains that shrink to nothing out of distribution. It constrains the loop on both sides, with a temporally annealed budget capping how many edits one candidate can bundle, and a selector carrying a critic that screens benchmark-specific proposals plus a pruner that deletes changes too small, too expensive or no longer useful. Across eight benchmarks it gains up to 14.1 points on the evolved split and still 4.7 points on five held-out benchmarks, and the resulting harness runs on 30% fewer policy tokens. Code is at github.com/google-research/rrsi.