CoPES post-trains a tool-using agent with evolution strategies at one-eighth the memory of full-parameter GRPO

arXiv·medium signalXBlueskyLinkedInCopy link

Posted 2026-08-03 (arXiv 2608.02391), CoPES targets the case where you have a few GPUs rather than a cluster. Evolution strategies avoid backpropagation and can eventually match gradient-based RL, but their GPU-hour cost makes them impractical on small setups, so CoPES decomposes the full parameter space into lower-dimensional subspaces searched cooperatively. Post-training a Qwen3.5-4B tool-using math agent under the GPU-hour budget of full-parameter GRPO's best validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain versus 67% for standard ES, with theoretical GPU memory under one-eighth of full-parameter GRPO, and beats standard ES and LoRA-based GRPO on all pass@k metrics across five benchmarks. Code is open-sourced.