SCX Router Scores Model Suitability With No Autoregressive Generation and a Persistent Text-Only KV Cache

arXiv 2609.02292·medium signalXBlueskyLinkedInCopy link

Inference endpoints differ widely in quality, price, latency, context support, tool use, domain expertise and reasoning behavior, which makes hand-maintained routing heuristics fragile. SCX Router is a GLiClass-based router that assigns a suitability score to each candidate model label without any autoregressive generation; the released 0.6B checkpoint pairs a Qwen3 decoder with a shallow bidirectional scorer, and its decoder-KV execution path keeps a text-only KV cache across a session, encoding only new dialogue turns and evaluating transient candidate-label tokens without adding them to the persistent cache. The same checkpoint also predicts task type, difficulty, reasoning mode and expected output length, and accepts custom zero-shot labels.