Qwen3.8-Flash-Next's n-gram table now streams off NVMe in SGLang.

A Hugging Face repo packages the model with the n-gram lookup table offloaded to SSD and streamed. The credible reply in the thread: a builder on an RTX Pro 6000 running the RAM variant reports over 12k prefill and over 170 tok/s single-stream decode, plus 440 tok/s at concurrency 4 on a 500W power-limited workstation. Another runs the SSD version on a 5090 with 64GB DDR5 at 34 tps and 55 pp, and swapped the q4 n-gram table for bf16 with no speed penalty and better output. The reason it works, per the thread, is that n-gram lookups are predictable enough to prefetch, so SSD latency stops mattering. (r/LocalLLaMA)