Qwen3.5 27B Pushed to 1.1M Tokens/Second on 96 B200 GPUs — Full vLLM Configs Published on GitHub

r/LocalLLaMA / r/singularity·medium signal

A practitioner achieved 1,103,941 tok/s with Qwen3.5 27B (dense) across 12 nodes with 96 NVIDIA B200 GPUs using vLLM. The 100x per-node speedup (9,500→95K tok/s) came from four changes: DP=8 over TP=8, context window reduction from 131K to 4K, chunked prefill tuning, and batch size optimization. At this throughput, 50,000 insurance policy documents process in hours instead of weeks, with sub-50ms per-token latency for 16K concurrent users. All configs are on GitHub.