Tip: Star Elastic's Elastic Budget Control Uses a Smaller Submodel for Thinking and Full Model for Answers — 16% Accuracy Gain at 1.9x Lower Latency

MarkTechPost·medium signal

NVIDIA's Star Elastic introduces elastic budget control — a novel inference scheme that routes the 'thinking' phase through a smaller nested submodel (12B or 23B) and switches to the full 30B model only for the final answer. This delivers up to 16% higher accuracy and 1.9x lower latency compared to standard budget control where the full model handles everything. The technique works because reasoning tokens are more tolerant of capacity constraints than answer-generation tokens. Applicable to any system with a think-then-answer architecture.