Fixed Reduction-Order GEMM Kernels Make LLM Linear Layers Bitwise Identical Across Ampere, Ada and Hopper, 1.17-3.1x Faster Than the Prior Fix

arXiv·medium signalXBlueskyLinkedInCopy link

The same model, prompt and stack give different greedy outputs on different GPUs because each architecture picks different matmul kernels with different reduction orders. Cooper, Jeong and Jeon use fixed-configuration fused-upcast GEMMs. They load 16-bit weights, accumulate in FP32, and use a reduction order that depends only on problem shape, so every GPU runs the same operation sequence. Linear-layer outputs were bitwise identical across NVIDIA Ampere, Ada and Hopper, ran 1.17-3.1x faster end-to-end than the state-of-the-art deterministic approach, and halved weight-memory traffic. This matters for anyone who needs replayable evals or agent traces across mixed GPU fleets.