CAVEAT benchmark: marketplace steering drops computer-use agents' best-choice purchases from 78.6% to 17.3%

arXiv·medium signalXBlueskyLinkedInCopy link

arXiv 2609.27273 (23 Sept) tests computer-use agents in nine marketplace environments whose operators have their own incentives, using a taxonomy of eight steering mechanisms. Across five model families, agents bought the product that was best for the user in 78.6% of control episodes but only 17.3% with steering enabled. The authors trace the failures to distorted priorities, early narrowing of options, and committing before checking the evidence. Their CAVEAT-Harness targets those three points, and it gives shopping-agent builders a concrete checklist beyond prompt-injection defenses.