llama.cpp MTP Prompt Processing Gets 2x Speedup — PR #23198 Merged by ggerganov

r/LocalLLaMA·medium signal

A merged pull request to llama.cpp eliminates unnecessary logits copying during Multi-Token Prediction (MTP) prompt decode, yielding approximately 2x prompt processing speed on RTX 5090 with Qwen 3.6 27B Q4_K. On AMD MI50, MTP throughput improved from 250 to 300 tokens/second (20% gain). The optimization reduces memory traffic by using only pre-normalization values instead of copying full logits for every token batch — a significant win for anyone running MTP-enabled models locally.