Fetching from the wire…
01
02
03
04
05
06
07
08
Models2026-09-21 · source-backed
mini-AGI trains from scratch on an RTX 3070 Laptop with 8GB VRAM using two dense prelude blocks plus one recurrent block applied up to 24 times, PonderNet per-character halting, top-8 MoE routing, and disk paging keeping only 32 experts resident so parameter count is bounded by disk rather than VRAM. At 318.1M characters it reports 0.8336 ± 0.0331 nats/char from a 540.1M-parameter pool of 169 experts at about 778 chars/second. The headline result is a hyperparameter one: setting the trunk learning rate to 0.1x the experts' rate dropped forgetting from +2.23 to +0.0067 nats after half a million characters of single-subject training.
Each link below shares sources, entities, or timing with this story.
Three signals hit in the same week. That's not coincidence, that's a threshold. First: Qwen 3.6-35B-A3B is running at 44 tokens per second on a single 16GB GPU at Q4 quantization with 100K context. It's a 35B-parameter MoE model with only 3B active parameters per token. A 293-...
JustVugg/colibri, Apache-2.0, 27,290 stars, treats storage, RAM and VRAM as one inference hierarchy with zero engine dependencies and one C file per model family. Eight families work today, including Kimi K3 at 2.8T and Inkling at 975B, all behind the same coli chat / coli ser...
An r/LocalLLaMA post reports a working deployment on 80x RTX 5090 connected over 25 gigabit Ethernet rather than NVLink or InfiniBand — roughly 2.5TB aggregate VRAM against a ~594GB MXFP4 weight file, surplus absorbed by activation and KV overhead. Expert-parallel MoE over com...
JustVugg/colibri is a dependency-free C engine treating VRAM, RAM, and storage as one memory hierarchy: ~17B dense params resident as int4 (~9.9GB), 19,456 routed experts (~19MB each, ~370GB total across 75 MoE layers) streamed from disk on demand. 14,728 stars since July 1 (~...
Google DeepMind released Gemma 4 on April 2 with four model sizes (E2B, E4B, 26B MoE, 31B Dense) under Apache 2.0. Multimodal (text, vision, audio). 256K context. Native thinking and tool-calling optimized for agentic workflows. Day-zero ecosystem support across vLLM, llama.cp...
A 135-upvote r/LocalLLaMA post catalogs them: an RFC for a MoE expert cache keeping hot CPU-resident experts in VRAM with hybrid hit/miss execution (#24528), AVX2 large-batch IQ prefill (#27402), tiled mul_mat for k-quants (#27851), AVX-512/VNNI paths for Q5_K/Q6_K (#27590), x...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.