arXiv: Agentic Transformers Provably Learn Depth-First Search via Reinforcement Learning

arXiv·medium signal

Yang, Huang, Liang, and Chi (arXiv:2606.00183, submitted May 29) prove theoretically that transformer-based agents can acquire depth-first search capabilities from sparse RL feedback alone. They construct a two-head transformer — one tracking prior actions, another detecting failures to trigger backtracking — and demonstrate via policy gradient under a depth-wise curriculum that DFS emerges in provable stages. This directly addresses the gap identified in 'Transformers Struggle to Learn to Search' (arXiv:2412.04703) with formal guarantees.