IBM Research's ALTK-Evolve beats ACE on AppWorld while using ~40% of its tokens — retrievable guidelines vs one giant playbook
Posted 2026-08-11 to the Hugging Face blog, IBM compares two agent-memory systems that learn from failures without weight updates. ACE builds a single playbook injected in full at every inference step; ALTK-Evolve consolidates lessons into individually retrievable guidelines with support counts and sends only a curated subset sized to each model's capacity. On AppWorld's 168 tasks with DeepSeek-V3.2: 89.3% goal completion at 263K tokens/task for ALTK-Evolve vs 80.4% at 634K for ACE. On the weaker gpt-oss-120b: 56.0% at 116K vs 54.8% at 777K — roughly one-seventh the inference cost. The practical takeaway for anyone building agent memory is that retrieval beats always-on context, and the gap widens as the model gets weaker.
↳ Follow the thread