Tip: llama.cpp CUDA Fast Walsh-Hadamard Transform Adds 7-9% Text Generation Speedup With Quantized KV Cache

GitHub·medium signal

A merged PR (#23615) to llama.cpp adds a CUDA-optimized Fast Walsh-Hadamard Transform (FWHT) kernel that delivers 1-2% prompt processing and 7-9% text generation speedup when using quantized KV cache (-ctk q8_0 -ctv q8_0). Benchmarked on an RTX 5090, the gains are free performance for anyone already running quantized KV cache on NVIDIA hardware. For local model users running coding agents, this is a meaningful throughput improvement with zero configuration changes.