S3Gym finds self-improvement is neither automatic nor uniform when agents test and judge their own behavior

arXiv·medium signalXBlueskyLinkedInCopy link

arXiv 2608.31100 (2026-08-31, 21 HF upvotes) points out that agent benchmarks evaluate models as fixed policies even though deployed agents accumulate behavioral experience. S3Gym separates permissive exploration from strict held-out evaluation across seven text-based games with executable environment verifiers, and tests three ways of folding experience back in: direct history in-context, score-conditioned summary memory, and parameter training. The finding that matters for anyone building a self-improving loop is the negative one, that gains do not appear reliably across the three pathways, so 'let it learn from its own runs' is not a design that works by default.