AWS documents how Motorway cut agent error rates from 1-in-8 to 1-in-50 with an eval pipeline

AWS Machine Learning Blog·medium signal

AWS Machine Learning published a production blueprint for evaluating AI agents using Strands and AgentCore, built with used-car marketplace Motorway. The end-to-end evaluation pipeline reduced incorrect results from 1 in 8 queries to 1 in 50 and cut issue detection from hours to minutes. Concrete before/after numbers on agent evals are rare — most vendor content stops at architecture diagrams.