Dream-RSI replays an agent's own discovery tree as a simulator so it can tune exploration policies off-policy

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

arXiv 2609.14858, submitted 2026-09-14 and the top HuggingFace daily paper at 239 upvotes, attacks the cost problem in recursive self-improvement: online policy optimization over exploration strategies needs long-horizon rollouts with delayed, expensive feedback. Dream-RSI keeps the coding agent unchanged and adds a thin orchestration layer that turns accumulated discovery history into a replay simulator, dreaming inside it to get cheap off-policy feedback before redeploying the improved policy online. It reports gains across algorithm engineering, mathematical optimization and GPU kernel engineering, and the pattern generalizes to any agent loop already logging its search tree.