An Open-Source Kernel Stack Got Qwen3.6-35B-A3B to 78,498 Output Tokens Per Second on 8 AMD MI350X, 2.16x vLLM

NetraRuntime, via r/LocalLLaMA·medium signalXBlueskyLinkedInCopy link

NetraRuntime published kernel-level optimizations for Qwen3.6-35B-A3B on AMD MI350X, reporting 11,161 output tok/s on one GPU and 78,498 tok/s mean (81,331 peak) across eight, which they measure at 2.16x vLLM throughput on the same 8-GPU benchmark. The kernels are open source at github.com/NetraRuntime/netra-kernel with a write-up of the method. Their closing note is the useful part for anyone attempting the same: once the kernels got fast enough, the bottleneck moved into scheduling, graph coverage, recurrent state, routing and HTTP serialization.