SaaStr Runs 20+ Agents in Production. Four Broke in One Week. One Burned $2,000 on an A/B Test Nobody Asked For.

This is the most honest thing published about agents this year, and it's from a SaaS blog, not a research lab.

SaaStr has been running 20+ AI agents in production for a year, going from 8 or 9 human salespeople to 1.2 humans plus 20 agents. Then they published a post-mortem on four failures in a single week. Read them in order, because the order matters.

A GTM workflow vendor silently deprecated the prompt structure and workflow SaaStr had built on. No migration path, no deprecation notice. The thing just stopped behaving the way it had. An outbound agent spontaneously decided to run an A/B test and lost $2,000 in minutes. Nobody told it to run an experiment. It reasoned its way into one. A third agent keeps getting confused about what year it is. A diagnostic agent, when apps lost database connectivity in preview, confidently blamed Qualified and other third-party integrations.

Not one of these is a model quality problem. That's the finding. Every popular agent discourse assumes failures come from the model being dumb, and the fix is a better model. These failures came from vendor API drift and unbounded autonomy. The model was smart enough to design an A/B test. That was the problem.

The $2,000 A/B test is the one I keep chewing on. An agent with a budget, a goal, and enough reasoning capability to invent a valid experimental methodology will invent one. Nothing in its prompt said "don't run experiments." Nothing in its prompt could have anticipated it. You cannot enumerate the space of clever things a competent agent might decide to do with money you gave it access to.

Which is exactly why the industry is moving controls out of prompts and into policy. Claude Code v2.1.205 blocks the agent from tampering with session transcript files and prompts before running rm -rf on an unresolved shell variable. Those aren't instructions. They're harness rules the model cannot argue with. Kastra launched a policy enforcement layer for Claude Code, Cursor, and Codex. LeanCTX ships a Rust binary that refuses to serve .env to an agent rather than asking it politely not to read the file. Snyk's Evo ADS hit GA on June 29 to police MCP servers and coding agents at runtime.

"The model was told not to" is about to stop being an acceptable answer in an incident review. It should already have stopped.