Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-07-27 · high
The build packs 2.5TB of VRAM against a 594GB model file, but nobody's posted tokens per second yet.
Why now: The post surfaced in the July 27 LocalLLaMA coverage as the first reported consumer-GPU run of Kimi K3.
A Reddit user got Kimi K3 running across 80 RTX 5090 GPUs, linked over standard 25 gigabit Ethernet, according to a post on r/LocalLLaMA. That matters because NVLink and InfiniBand are the expensive part of any GPU cluster built for models this size. If Ethernet holds up, local-LLM builders get a cheaper path to running huge mixture-of-experts models without buying specialized interconnect hardware.
The numbers back up the scale. Eighty 5090s add up to roughly 2.5TB of aggregate VRAM, against a ~594GB MXFP4 weight file for K3. The leftover room covers activation memory and KV cache overhead, margin most local builds don't have.
Kimi K3 is a mixture-of-experts model. Expert-parallel MoE over commodity Ethernet is exactly the setup vLLM's engineering blog has warned about. Network bandwidth, not compute, ends up capping how fast any single user gets tokens back.
What's missing is a tokens-per-second number. The post is a single, unaudited report that nobody else has replicated yet.
Getting a 594GB model to load across 80 cards over Ethernet is real. Whether it's usable depends on throughput, and throughput is the one number missing from the thread. An 80-GPU Ethernet cluster and an 80-GPU Ethernet paperweight look identical without it.
Each link below shares sources, entities, or timing with this story.
Google DeepMind released Gemma 4 on April 2 with four model sizes (E2B, E4B, 26B MoE, 31B Dense) under Apache 2.0. Multimodal (text, vision, audio). 256K context. Native thinking and tool-calling optimized for agentic workflows. Day-zero ecosystem support across vLLM, llama.cp...
deltafin, created July 28 and already at 243 stars, streams MXFP4 expert weights on demand over HTTP into a local disk cache rather than resident memory, with fused NEON kernels, Metal/MPS compute, exact reproducible decoding, and an OpenAI-compatible server. This directly con...
Three signals hit in the same week. That's not coincidence, that's a threshold. First: Qwen 3.6-35B-A3B is running at 44 tokens per second on a single 16GB GPU at Q4 quantization with 100K context. It's a 35B-parameter MoE model with only 3B active parameters per token. A 293-...
Alibaba released Qwen3.6-27B on April 22. Dense architecture. Open weights. 77.2% on SWE-bench Verified, within 3.7 points of Claude Opus 4.6. On SkillsBench, it scores 48.2% versus its own 397B MoE predecessor's 30.0%. That's a 77% improvement with 14.8x fewer parameters. Let...
A developer published v100-skinny with hand-written NVFP4 W4A16 CUDA kernels plus chain-MTP speculative serving: four V100s at 219.1 ± 5.9 tok/s decode against a 5090 running NInfer at 214.7 ± 9.2, both 5/5 correct on AIME 2026 problem 1 across five seeds. (r/LocalLLaMA) The m...
The file is 1.56TB. That's the first thing you notice about the moonshotai/Kimi-K3 Hugging Face repo that went live today. 2.8 trillion total parameters, 104B activated, 896 routed experts with 16 selected plus 2 shared per token, 93 layers split 69 Kimi Delta Attention and 24...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.