Route 85% of Your Queries to Cheap Models and Lose Almost Nothing

UC Berkeley and Canva published production results on RouteLLM, and the numbers are hard to argue with: 85% cost reduction on MT Bench while retaining 95% of GPT-4 quality. The practical pattern is an 85/10/5 budget split. 85% of queries go to budget-tier models. 10% to mid-tier. 5% to frontier.

I've been doing manual routing in my own pipeline (sending different research agents to different models based on task complexity), but I haven't formalized it the way this research suggests. The case study that got my attention: a document processing pipeline dropped per-document cost from $1.40 to $0.34 by routing 4 of 7 agent steps to mid-tier and 2 to small models. Only the final synthesis step hit the frontier model.

This pairs directly with the harness story above. If 98.4% of your agent is deterministic infrastructure, then the routing decision, which model handles which step, is the single most important economic choice your harness makes. And most builders aren't making it at all. They're sending everything to the same model regardless of task difficulty.

The AICC enterprise token cost report confirms the macro picture: enterprise token costs fell 67% year-over-year through April 2026, driven by open-source pricing pressure (DeepSeek V4, Qwen 3.6-Plus) and multi-model routing adoption. But here's the catch. Agentic AI token consumption runs 5-30x higher per task than chatbot interactions. The per-token savings are being eaten alive by volume growth.

So you need both: cheaper tokens AND smarter routing. The teams that figure this out first will run agent workloads that their competitors literally can't afford to match.

Actionable advice: take your current agent pipeline and categorize each step as "needs reasoning" or "needs execution." Reasoning steps (planning, synthesis, evaluation) stay on frontier. Execution steps (formatting, extraction, classification) go to budget models. Start with Anthropic's own model routing or open-source RouteLLM. Measure quality on a per-step basis, not just end-to-end.

The 85/10/5 split isn't a suggestion. It's a survival strategy as token volumes compound.