Direct-P reaches 2.13x BF16 attention throughput on a GB200 by skipping softmax conversion.

Blackwell's FP4 tensor cores don't automatically speed up attention, because softmax conversion and on-chip dependencies dominate once the matrix products shrink. Direct-P maps scores directly to FP4 probabilities for noncausal inference. A separate causal path reconstructs probabilities from saved quantized queries and keys and uses FP8 gradient operands, accelerating a full single-GPU 8B update by up to 1.14x. The caveat is sharp: matched distributed training must retain FP8 probabilities and values, because every tested MXFP4 probability/value training trajectory diverged. arXiv 2609.04105