Fetching from the wire…
Infra2026-09-15 · source-backed
PR #52781 fixes prepare_finalize/deepep_v2.py issuing the ElasticBuffer combine synchronously at both call sites, which meant the modular-kernel shared-expert overlap window was never used and the shared-expert FFN serialized behind the cross-node all-to-all on every MoE layer during decode (GitHub). finalize_async now issues the combine with async_with_compute_stream and returns a receiver closure joining via a device-side event wait, which is graph-replay safe. On DeepSeek-R1-671B FP8 at EP8 across two GB300 nodes with full cudagraphs, decode ITL p50 goes 15.71 ms to 14.32 ms at concurrency 1 and 22.05 to 20.89 ms at concurrency 24, greedy outputs bit-identical across four independent ABBA server starts. Kill switch is VLLM_DEEPEP_V2_COMBINE_OVERLAP=0.
Each link below shares sources, entities, or timing with this story.
The September 5 release adds beam search via a beam_width request parameter returning the n best sequences, though it doesn't yet combine with speculative decoding, disaggregation, DP attention or HiCache. DeepEP v2's fixed-size ElasticBuffer engine as --moe-a2a-backend deepep...
The abliteration tool gained 215 stars to reach 30,103, but the stronger signal is downstream: the HF trending endpoint returns DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU and Momoking/Qwen3-VL-32B-Heretic-MiniMax-H3-NVFP4, both naming the too...
PR #26622 pushes a user-specified number of FFN sublayers to CPU while attention stays on the GPU, mirroring --n-cpu-moe (GitHub). The practical note from the r/LocalLLaMA thread is that the previous route was regex matching in -ot style, and for a model you plan to run for mo...
78 layers where layer one is dense FFN and the other 77 are MoE, each with 256 routed experts and 1 shared expert, top-8 routing per token, plus a native 10B MTP layer (0.7B activated) built in for speculative decoding. FP8 and base variants released together on August 28; the...
MIT, Swift and MLX, created August 28, 220 points on Hacker News (GitHub). It runs Qwen3.8-Flash-Next, 103.8GB across 24 files at 4-bit, by loading only the 3.8GB dense trunk into RAM (about 2 second startup) and reading routed experts with pread into a fixed pool of cache slo...
Released August 28 with 78 layers, 77 of them MoE with 256 routed plus one shared expert and top-8 routing, plus a native 10B MTP layer for speculative decoding (GitHub). The attention stack uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reus...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.