Fetching from the wire…
01
02
03
04
05
06
07
08
Models2026-09-05 · source-backed
A llama.cpp fork adds Q2_B3, packing ternary weights directly in base 3: 128 weights per block become 26 bytes of trits plus one f16 scale, 28 bytes total, 1.75 bits per weight. On genuinely ternary models like BitNet-b1.58 or Ternary-Bonsai, a 9B goes from about 2.5GB to 2.0GB and a 27B from 7.6GB to 5.9GB, weights only. Stated caveats matter: this is not a general 2-bit quantizer, feeding it an FP16 model destroys quality, and the CUDA path compiles but is unverified because the author only owns a 7900 XTX (r/LocalLLaMA).
Each link below shares sources, entities, or timing with this story.
PR #24448 adds Q2_0 to ggml for CPU (ARM NEON plus scalar fallback), completing the Q1_0/Q2_0/Q4_0/Q8_0 family, primarily to serve PrismML's Apache-2.0 Ternary Bonsai models. Format packs 2 bits per weight with one fp16 scale per 64 weights mapping {0,1,2,3} to {-1,0,+1,+2}·d....
The file is 1.56TB. That's the first thing you notice about the moonshotai/Kimi-K3 Hugging Face repo that went live today. 2.8 trillion total parameters, 104B activated, 896 routed experts with 16 selected plus 2 shared per token, 93 layers split 69 Kimi Delta Attention and 24...
PR #19378 landed in llama.cpp this week, and I think most people are underselling what it means. Backend-agnostic tensor parallelism via --split-mode tensor makes multi-GPU inference work across AMD, Intel, and Apple Silicon. Not just CUDA. Everything. For context: llama.cpp h...
Can a model that fits on a Raspberry Pi do reliable tool calling? Two independent labs just answered yes. PrismML emerged from stealth March 31 with Bonsai, the first commercially viable 1-bit LLMs built on Caltech research. The 8B model fits in 1.15GB (vs 16GB for FP16), runs...
A builder fine-tuned Gemma 4 12B specifically for tool use and CLI work because nothing larger fits comfortably in 16 GB, reporting 2.7x on tool calling plus a 15.7% rise in tool calls attempted, which they read as less time lost in reasoning. Weights published fp16 through Q4...
For about a year, "run your agent locally" meant accepting a model that couldn't reliably call a tool twice in a row. That excuse is gone. Meta Superintelligence Labs published Muse Glimmer today: a 29.6B dense causal transformer, 52 layers, 6,656 hidden dim, with a ~1.8B ViT-...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.