Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-29 · source-backed
A builder posted a full recipe reaching 181 tok/s aggregate across about 9 concurrent agent sessions on Qwen3.8-Flash-Next NVFP4 at 512K context, nodes joined by direct ConnectX-7 over NCCL/RoCE at 200 Gb with TP=2. The warning in the post is that TCP fallback is silent and costs half the speed, so you must confirm "Using network IB" in the NCCL log. Mapping the 47.7 GiB FP8 n-gram table off NVMe dropped per-node weights from 65 to 41 GiB, and two fixes made it fast: madvise(MADV_RANDOM) killed a 30x read amplification, taking one 405K prefill from 603 GB of disk reads down to 19 GB, and 64 gather threads removed fault-latency serialization, freeing memory for a 2.89M-token KV pool. (r/LocalLLaMA)
Each link below shares sources, entities, or timing with this story.
Google released Flash / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (Google released Flash); both cover Flash, Qwen3; reported by the same outlet (reddit.com).
Shared entities / Same source domain / Shared topic / Earlier coverage
Both cover Flash, GiB, LocalLLaMA; reported by the same outlet (reddit.com); overlapping topics (context, cost).
Shared entities / Same source domain
Both cover Flash, LocalLLaMA, NVMe, Qwen3; reported by the same outlet (reddit.com).
Google released Flash / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Google released Flash); both cover LocalLLaMA, Using; overlapping topics (context, cost).
Google released Flash / Shared entities / Same source domain / Earlier coverage
Linked by a graph relationship (Google released Flash); both cover LocalLLaMA, Qwen3; reported by the same outlet (reddit.com).
Shared entities / Same source domain / Shared topic / Earlier coverage
Both cover Flash, LocalLLaMA; reported by the same outlet (reddit.com); overlapping topics (agent, context, cost).
Shared entities / Same source domain / Earlier coverage
Both cover Flash, LocalLLaMA, Qwen3; reported by the same outlet (reddit.com); earlier Flash coverage from 2026-08-27.
Both cover Flash, LocalLLaMA, Qwen3; reported by the same outlet (reddit.com); earlier Flash coverage from 2026-08-10.