60% MatMul Performance Bug in cuBLAS on RTX 5090 — Inefficient Kernel Dispatched for All Batched FP32

r/MachineLearning·high signal

A researcher demonstrated that cuBLAS dispatches the tiny `simt_sgemm_128x32_8x5` kernel for every batched FP32 workload from 256×256 to 8192×8192×8 on RTX GPUs, achieving only ~40% FMA pipe utilization. A custom 300-line implementation hits ~68% utilization on Blackwell, outperforming cuBLAS by 40–60% on affected workloads. Likely affects all RTX SKUs, not just the 5090. Critical for anyone running local inference with batched operations — check if your workload is hitting this path.