Fetching from the wire…
Infra2026-09-14 · source-backed
The v1.11.0 release reads DeepSeek V4.1 Flash's vendor checkpoint natively with no conversion, 510 GB on disk, fp8 dense with 32x32 ue8m0 tiles and fp4 experts byte-identical to the mxfp4 its Kimi K3 engine already reads. A cold turn went from 78.7s to 25.1s through batched expert reads (default depth 8 is the measured knee), reading attention matrices once per block of positions, and an expert-major MoE. Five-turn chat runs 1.14-1.58 tok/s. Two attempts at hiding expert reads behind matmuls were built, measured worse and removed, with the numbers written into docs/deepseek-v41.md so the next attempt starts from them. More projects should do that.
Each link below shares sources, entities, or timing with this story.
JustVugg/colibri, Apache-2.0, 27,290 stars, treats storage, RAM and VRAM as one inference hierarchy with zero engine dependencies and one C file per model family. Eight families work today, including Kimi K3 at 2.8T and Inkling at 975B, all behind the same coli chat / coli ser...
The file is 1.56TB. That's the first thing you notice about the moonshotai/Kimi-K3 Hugging Face repo that went live today. 2.8 trillion total parameters, 104B activated, 896 routed experts with 16 selected plus 2 shared per token, 93 layers split 69 Kimi Delta Attention and 24...
An open-weight Chinese frontier model is now a dropdown option in Microsoft's coding product. That happened before anyone finished characterizing what the model does. GitHub's changelog dated August 6 makes Kimi K3 generally available across Copilot Pro, Pro+, Max, Business an...
Hy4-Preview runs 256 routed experts plus one always-active shared expert with top-8 routing, combining Multi-head Latent Attention, DeepSeek Sparse Attention with shared indexer layers, gated MLA with learnable attention sinks, and Independent Hyper-Connections replacing the p...
DeepSeek dropped V4 in mid-June as an open-weight model with a 1-million-token context window, priced at $1.74 per million input tokens, posting near-parity with GPT-5.4 on math and Q&A benchmarks (MindStudio). That's the headline number. The architecture underneath is more in...
sqliteai/waste (769 stars, created July 28) keeps only K3's 27.28GB trunk resident and streams the ~4% of experts activated per token off SSD, opening the undistilled model in 29.06GB of RAM. The counterintuitive result is the cache table: raising the expert cache from 17.32GB...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.