Agentic ESOpt Fine-Tunes Full-Parameter Long-Horizon Agents With Only Inference-Level GPU Memory
arXiv 2608.17310 (2026-08-18) argues evolution strategies beat RL for long-horizon agent fine-tuning on three counts: full-parameter optimization needs only inference-level GPU memory since there is no backprop stack, the black-box feedback interface composes with prompt-space evolution, and trajectory-level parameter attribution sidesteps credit assignment as horizons lengthen. Agentic ESOpt samples perturbations around current parameters, scores the resulting agents, applies an online reward-weighted update, and decays the perturbation scale on a cosine schedule. Full-parameter optimization of Qwen-3.5-27B improves the no-skill baseline by 6.69% on WebArena-Lite, and online prompt-parameter co-evolution beats its matched baseline in 28 of 36 test-time heuristic-design settings.
↳ Follow the thread