Route requests to the cheapest capable model by task difficulty before reaching for caching or batching

Mavik Labs·medium signal

The 2026 LLM cost stack layers routing, caching, and batching to cut 70–85% of spend, and routing is the highest-leverage lever: classify each request's difficulty and send easy ones to a small model, escalating to a frontier model only when needed. Most production traffic is easy queries being over-served by a top-tier model. Add a lightweight difficulty classifier in front of your LLM calls and you reclaim the largest share of the bill before touching prompt hygiene.