'Resample or Reroute?' Reframes LLM Routing Under Real Budget Constraints

arXiv·medium signal

LLM routers trade response quality against serving cost, but the reported headroom over a per-instance oracle assumes correctness labels and an unconstrained budget that no deployed system has. This work studies budget-aware test-time model selection, showing test-time resampling can recover per-instance selection headroom that a single-commit router misses. Relevant to anyone building cost-aware model-routing layers across a fleet of models.