Harvey's gross margin went from +50% to -50% on token spend, then came back on a Kimi K3 fine-tune

This is the first time I've seen a company publish the whole arc: the agent upgrade, the cost explosion, the margin collapse, and the specific fix.

Bloomberg reports that Harvey, the legal AI company, released an agent update in March 2026. Token consumption went up twentyfold. Not 20%. Twenty times. Under the usage-based enterprise pricing Harvey has with OpenAI and Anthropic, that flowed straight through to cost of goods sold. Gross margin was around 50% in January. By June it was negative 50%. A company selling software to law firms was paying more for inference than it charged for the product. (Bloomberg)

The fix took five months and it wasn't prompt engineering. Harvey post-trained its own model on Moonshot's open-weight Kimi K3 and routed most work there, keeping Claude Opus for the hardest tasks. Margins turned positive again in August. Bloomberg names three more companies doing versions of the same thing: Abridge is building a clinical foundation model on NVIDIA's open model, and Ramp and Rogo are exploring training their own for the first time.

What makes this the most useful story in today's set is that it's a mechanism, not a mood. The agent update didn't make the model more expensive. It made the task more expensive, by running more turns, reading more context, and retrying more. Per-token pricing was flat the whole time. The bill quadrupled anyway.

If you're building anything agentic on a usage-based contract, the number you need on a dashboard is dollars per completed task, broken out by task type, tracked as a time series across releases. Not tokens per call. Not average cost per request. Per-task, per-release. Harvey went five months before the line crossed zero, which means the signal existed for months before anyone acted on it.

The second lesson is about the escape hatch. Harvey didn't negotiate a better rate. It moved the volume work onto open weights it controls and kept the frontier model for the top of the difficulty distribution. That's a real architecture, and it requires you to have measured which of your tasks are actually hard. Most teams haven't. They route everything to the best model because it's one line of config, and they find out what that costs when the invoice arrives.

Three months ago the open-weight argument was mostly about sovereignty and vendor lock-in. Now it's a CFO argument with a named company and a signed number attached.