Fetching from the wire…
01
02
03
04
05
06
07
08
Models2026-09-04 · source-backed
Supports both an integrated MTP head and a separate -md draft file, with the poster reporting 45 to 90 tok/s on a 5090 with 128GB system RAM and working configs down to a 12GB 4070. A separate 2x3090 plus DDR4 build reports 37-41 t/s decode with UD-Q4_K_XL plus expert cache plus MTP, up from 25-29 t/s two days earlier. Both are single-builder measurements on hand-built branches. r/LocalLLaMA
Each link below shares sources, entities, or timing with this story.
A Hugging Face repo packages the model with the n-gram lookup table offloaded to SSD and streamed. The credible reply in the thread: a builder on an RTX Pro 6000 running the RAM variant reports over 12k prefill and over 170 tok/s single-stream decode, plus 440 tok/s at concurr...
PR 27742 adds gated delta net layers, 512-expert MoE with top-10 selection, hyper-connections, query-key sparse attention and the per-layer n-gram embeddings as a mmap table that can sit in RAM or on disk (GitHub). Reported 55 tok/s on 4x3090 with the Q4 GGUF, and one commente...
The repo appeared August 24, opening the weights of a multimodal MoE that Qwen frames explicitly as an architecture preview, the same role Qwen3-Next played for Qwen3.5 (GitHub). The hybrid Gated DeltaNet plus Gated Attention design it previews already carried through the Qwen...
Unsloth's UD-Q2_K_XL (78.9 GB) plus a 358,400-token slot via YaRN from the native 262,144 with fp16 KV fits under the default 96GB Metal wired limit, no sysctl hack. Cold prefill runs 1,561 t/s at 5.6K context down to 318 t/s at 111K; a normal incremental turn is 77-854 t/s ou...
MIT, Swift and MLX, created August 28, 220 points on Hacker News (GitHub). It runs Qwen3.8-Flash-Next, 103.8GB across 24 files at 4-bit, by loading only the 3.8GB dense trunk into RAM (about 2 second startup) and reading routed experts with pread into a fixed pool of cache slo...
A month of solo work produced quants for LongCat-Flash-Lite-Sparse, Qwen3.8-27B, Qwen3.5-122B-A10B with MTPs, Qwen3-Coder-Next and Laguna-S2.1 with vision. Getting there meant writing Heretic support for the architecture from scratch and then adding llama.cpp support, and main...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.