AgentRouter routes each trajectory step to one of four model tiers with a 12M-parameter classifier

arXiv·medium signalXBlueskyLinkedInCopy link

The authors argue enterprise agentic systems that send every trajectory step to a frontier model waste 60-80% of their inference budget, because subtask complexity varies inside a single trajectory: a planning step needs frontier reasoning while the formatting step after it needs a 7B model. AgentRouter formalizes this as sequential assignment and trains a 12M-parameter classifier on 50,000 annotated agent trajectory steps spanning planning, coding, research and data analysis, using five features available at routing time with under 5ms overhead per step on an A100. This is the step-level counterpart to the query-level routing everyone already ships, and the overhead number is small enough to be worth trying in a real harness.