Nvidia's Blackwell Ultra Just Made Self-Hosted Inference a Real Option: 2.49M Tokens/Sec, 30 Cents Per Million

Nvidia set new MLPerf Inference v6.0 records on April 2 using four GB300 NVL72 systems (288 Blackwell Ultra GPUs) interconnected via Quantum-X800 InfiniBand. The headline number: 2.49 million tokens per second on DeepSeek-R1 in offline mode. That's the largest GPU configuration ever submitted to any MLPerf benchmark.

The number that matters more for builders: 250K tokens/sec on the interactive benchmark at 30 cents per million tokens generated. That's a 2.77x speedup over the prior-generation GB200 NVL72.

Nvidia was the sole platform to submit across all new tests, including Qwen3-VL-235B and text-to-video generation. Nobody else could even run the full suite.

I keep coming back to the 30 cents number. Right now, if you're calling Claude or GPT APIs at scale, you're paying somewhere between $3 and $75 per million output tokens depending on the model. Self-hosted inference on Blackwell Ultra at $0.30/M is an order of magnitude cheaper than most API pricing. Yes, the upfront hardware cost is enormous. Yes, you need the expertise to run it. But for companies processing millions of requests daily, the build-vs-buy math just shifted hard.

This also matters for the open model ecosystem. vLLM just crossed 75K stars with expanded Blackwell support. NVIDIA is optimizing Gemma 4 for deployment across RTX to DGX Spark to Jetson. The inference stack is maturing fast enough that "run your own models" is becoming a real option for mid-size companies, not just hyperscalers.

If you're planning inference infrastructure for the next 12 months, these benchmarks are your baseline. The 30-cent floor reshapes every cost model I've seen.