EnvACE trains tool-use agents by making the policy rehearse the environment instead of calling one

arXiv·medium signalXBlueskyLinkedInCopy link

Xu et al. (arXiv 2608.06197, submitted August 6) replace external environment interaction during agentic RL with world rehearsal: the policy alternates between emitting a tool call and playing the environment to produce that call's response, then conditions later decisions on the rehearsed response, with both roles optimized end-to-end on task-success rewards. This sidesteps the cost of building and verifying executable environments, and the internalized world model also lets the agent privately rehearse at test time before committing an execution, yielding further gains under a moderate rehearsal budget. It outperforms environment-scaling baselines overall across BFCL-v4, tau^2-Bench, VitaBench and FinMCP-Bench, with code at github.com/Within-yao/EnvACE.