An Agent Marketplace Guardrail Study's +87 Welfare Gain Collapses to +7 Once the Protocol Is Held Fixed

arXiv 2609.01519·high signalXBlueskyLinkedInCopy link

Auditing a multi-turn buyer-seller testbed of LLM agents, the authors show an initial implementation reported welfare gains of +87.4, +35.0, and +28.8 across a Qwen2.5 1.5B-14B ladder while giving guarded and unguarded agents different offer schemas and choice procedures. Holding the schema and buyer chooser fixed moves the paired contrasts to +7.2, -13.9, and +23.8; the four largest 14B single-generation effects averaged +229 but fell to +37.6 with a 95% bootstrap interval of [-34.2, 109.3] once averaged over three generations, with generation residuals accounting for 49.9% of variation. They propose a construct-validity contract that returns INVALID or INCONCLUSIVE before any policy claim is made.