Top 5 · 2026-04-03 · source-backed
Nvidia's Blackwell Ultra Just Made Self-Hosted Inference a Real Option: 2.49M Tokens/Sec, 30 Cents Per Million
Story
Nvidia set new MLPerf Inference v6.0 records on April 2 using four GB300 NVL72 systems (288 Blackwell Ultra GPUs) interconnected via Quantum-X800 InfiniBand. The headline number: 2.49 million tokens per second on DeepSeek-R1 in offline mode. That's the largest GPU configuration ever submitted to any MLPerf benchmark.
The number that matters more for builders: 250K tokens/sec on the interactive benchmark at 30 cents per million tokens generated. That's a 2.77x speedup over the prior-generation GB200 NVL72.
Nvidia was the sole platform to submit across all new tests, including Qwen3-VL-235B and text-to-video generation. Nobody else could even run the full suite.
I keep coming back to the 30 cents number. Right now, if you're calling Claude or GPT APIs at scale, you're paying somewhere between $3 and $75 per million output tokens depending on the model. Self-hosted inference on Blackwell Ultra at $0.30/M is an order of magnitude cheaper than most API pricing. Yes, the upfront hardware cost is enormous. Yes, you need the expertise to run it. But for companies processing millions of requests daily, the build-vs-buy math just shifted hard.
This also matters for the open model ecosystem. vLLM just crossed 75K stars with expanded Blackwell support. NVIDIA is optimizing Gemma 4 for deployment across RTX to DGX Spark to Jetson. The inference stack is maturing fast enough that "run your own models" is becoming a real option for mid-size companies, not just hyperscalers.
If you're planning inference infrastructure for the next 12 months, these benchmarks are your baseline. The 30-cent floor reshapes every cost model I've seen.
Related stories
Each link below shares sources, entities, or timing with this story.
Shared entities / Same source domain / Shared topic / What happened next
Gemma 4 Ships Under Apache 2.0 and the Numbers Are Absurd
Both cover April, CLAUDE, Gemma, GPU; reported by the same outlet (github.com); overlapping topics (inference, model, number).
Shared entities / Same source domain / Shared topic / What happened next / Downstream implication
llama.cpp Gets Backend-Agnostic Tensor Parallelism, and the NVIDIA Tax Just Got Optional
Both cover Blackwell, GPU, NVIDIA, Qwen3; reported by the same outlet (github.com); overlapping topics (inference, matter, model, nvidia).
Shared entities / Shared topic / What happened next / Tension
A 27B Model Just Matched the 400B One. It Fits on Your Laptop.
Both cover April, GPU, Qwen3, RTX; overlapping topics (cost, model, number, token); picks up the April thread on 2026-04-23.
Shared entities / Same source domain / Shared topic / What happened next / Tension
GPU vector indexing is now the default in OpenSearch Serverless, 10x faster at a quarter the cost.
Both cover Blackwell, GPU, NVIDIA; reported by the same outlet (blogs.nvidia.com); overlapping topics (blackwell, cost, nvidia).
Shared entities / Shared topic / What happened next
DeepSeek V4 Drops: MIT-Licensed Frontier Model at 1/6th the Cost, Running on Chinese Chips
Both cover April, Claude, DeepSeek, NVIDIA; overlapping topics (cost, model, token); picks up the April thread on 2026-04-26.
Xiaomi Just Open-Sourced a 1-Trillion Parameter Model Under MIT License. It Built a Compiler in 4 Hours.
Both cover April, DeepSeek, Gemma; overlapping topics (benchmark, cost, matter, model, token); picks up the April thread on 2026-05-01.
Shared entities / Shared topic / Earlier coverage / Tension
NVIDIA Vera Rubin Enters Full Production — 10x Cost-per-Token Improvement
Both cover Blackwell, GPU, NVIDIA; overlapping topics (blackwell, inference, nvidia); earlier Blackwell coverage from 2026-02-22.
Shared entities / Same source domain / Shared topic / What happened next
Claude Code's UX Won. Now People Are Swapping the Brain.
Both cover Claude, Qwen3; reported by the same outlet (github.com); overlapping topics (cost, matter, model, token).
Source trail
Entities
Provenance
- Canonical issue
- Ramsay Research Agent — April 3, 2026
- AI generated
- no
- Story unit
- 2026-04-03-nvidia-s-blackwell-ultra-just-made-self-hosted-inference-a-real-option-2-49m-tokens-sec-30-cents
- Labels
- source-backed, canonical briefing excerpt