Reddit
Speculative Decoding Achieves 665% Speed Increase in llama.cpp — Community Investigates Why
A r/LocalLLaMA user reported a 665% speed increase using speculative decoding with llama.cpp's ngram-map-k settings (--spec-ngram-size-n 24, --draft-min 12, --draft-max 48), specifically for code modification prompts where high token predictability enables aggressive speculation. The 63-upvote post with 37 comments (0.59 comment-to-score ratio) drew deep technical discussion about why certain prompt patterns achieve much higher acceptance rates. For builders running local models for code tasks, these settings represent a practical 6x speedup on specific workloads.
Source
↳ Follow the thread