ACWorld benchmarks "vibe commerce" with independently controlled buyer and merchant agents over 785,022 real listings

arXiv·medium signalXBlueskyLinkedInCopy link

Posted 2026-08-03 (arXiv 2608.02441) by a 12-author group including Julian McAuley, Manling Li, and Philip S. Yu, Agentic Commerce World models the case where buyer and merchant agents hold private objectives and distinct authority in a shared market. Its Vibe Commerce Protocol validates agent actions before updating shared transaction state and records the interaction, making runs auditable and reproducible rather than judged on final state alone. The benchmark has a 200-task capability-coverage track and a 60-task large-catalog track over 785,022 transactable listings; across ten models mean scores span 65.9% to 85.6% and 56.1% to 91.4%. The headline methodological result is that final state alone misses real errors and incomplete trajectories still carry useful process signal.