kernel-anvil: 2x Decode Speedup on AMD GPUs by Auto-Tuning llama.cpp Kernels Per Model Shape
r/LocalLLaMA·medium signal
A new tool called kernel-anvil profiles GGUF model layer shapes on AMD GPUs and generates optimal kernel configurations that llama.cpp loads at runtime — no recompilation needed. The problem: llama.cpp's default Metal/HIP kernels use one-size-fits-all configs, leaving significant performance on the table for AMD hardware. The tool auto-tunes per-layer kernel parameters to match the specific model architecture, claiming 2x decode speedup on AMD GPUs. High comment-to-score ratio (39 comments on 86↑ = 0.45) indicates deep technical engagement. Reddit: r/LocalLLaMA (86↑, 39 comments).