Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-26 · high
NetraRuntime says the open source kernels run 2.16 times faster than vLLM on the same benchmark.
Why now: NetraRuntime says its kernel work is done, so the scheduling and serving bottleneck it flags is the next problem for teams running inference on AMD MI350X GPUs.
NetraRuntime's kernel stack pushed Qwen3.6-35B-A3B to a mean of 78,498 tokens a second across eight AMD MI350X GPUs, peaking at 81,331, per NetraRuntime's benchmark writeup. That gap matters for anyone deciding how many MI350X GPUs a workload needs. The same kernels reach 11,161 tokens a second on a single GPU, 2.16x vLLM's throughput on the identical benchmark. A team can reach the same throughput target with fewer GPUs once the kernel stack replaces the default one.
The kernels are open source, so teams running the model on AMD hardware can rerun the numbers instead of taking NetraRuntime's word for it. Running eight GPUs together adds scheduling and routing overhead that a single-GPU test doesn't see. NetraRuntime's own writeup says that overhead is where its kernels now run out of room.
The writeup names where the bottleneck moved once the kernel code got fast enough: scheduling, graph coverage, recurrent state handling, routing, and HTTP serialization. Fixing the fast part and watching the slow part become the next bottleneck is the shape of every optimization project I've finished.
Each link below shares sources, entities, or timing with this story.
Shared entities / Earlier coverage
Both cover A3B, GPU, Qwen3; earlier A3B coverage from 2026-07-27.
Shared entities / Earlier coverage / Tension
Both cover GPU, Qwen3; earlier GPU coverage from 2026-06-20; pushes against this story (versus).
Both cover GPU, Their; earlier GPU coverage from 2026-04-30; pushes against this story (but).
Both cover GPU, Qwen3; earlier GPU coverage from 2026-04-23; pushes against this story (versus).
Shared entities / Earlier coverage / Downstream implication
Both cover GPU, Qwen3; earlier GPU coverage from 2026-04-10; traces where this leads (what it means).
Shared entities / Earlier coverage / Tension
Both cover A3B, Qwen3; earlier A3B coverage from 2026-03-22; pushes against this story (vs).
Shared entity: Qwen3 / Shared topic / Earlier coverage / Tension
Both cover Qwen3; overlapping topics (benchmark, measured); earlier Qwen3 coverage from 2026-08-25.
Shared entity: GPU / Shared topic / Earlier coverage / Tension
Both cover GPU; overlapping topics (benchmark, kernel); earlier GPU coverage from 2026-06-20.