TAPS: Task-Aware Speculative Decoding Shows Specialized Draft Models + Confidence Routing Beat Generic Approaches (108 HF Upvotes)

arXiv / HuggingFace Daily Papers·medium signalXBlueskyLinkedInCopy link

arXiv 2603.27027 (March 27, top trending on HuggingFace with 108 upvotes) demonstrates that speculative decoding performance depends heavily on matching draft training data to downstream workload. Math-trained drafters (MathInstruct) dominate reasoning benchmarks while ShareGPT-trained drafts excel on general-purpose tasks (MT-Bench), and confidence-based routing between specialized drafters outperforms both entropy-based routing and naive weight-space merging. For anyone running local inference at scale, this means maintaining 2-3 task-specialized draft models with a confidence router beats one universal drafter.