Fetching from the wire…
Public story · 2026-09-08 · high
A zero-filled test passed while a nonzero one failed by 2.35e-6, exposing a rollback bug in Kimi-K3 support that a weaker test would have missed.
Why now: The fix is open in llama.cpp's repository as of September 8, 2026.
Kimi-K3 gained speculative-decoding rollback in llama.cpp's PR #28466, and testing it exposed a state bug that had stayed hidden.
The fix matters because getting this wrong doesn't crash llama.cpp, it corrupts output silently. Without the right snapshots, a model running Kimi-K3 with rollback enabled would restore stale memory after a rejected draft, invisibly.
Speculative decoding drafts tokens ahead of the full model, then rejects the ones that don't match. For architectures with recurrent state, like Kimi-K3's KDA layers, rejecting a draft means rolling that state back too.
Kimi-K3 was added to the llm_arch_supports_rs_rollback allowlist without the storage to make that rollback safe. It only kept the model's final KDA state, not a snapshot at each rollback position. Flipping the allowlist alone, without snapshot writes, would have restored unwritten state groups with whatever was already sitting in memory after a rejected draft.
The PR saves convolution windows per rollback position along with KDA state snapshots, so a rollback restores the correct state instead of stale data.
The test note shows why the bug stayed hidden until now. Run the allowlist change alone against a zero-filled state and the split-replay check passes clean. Run it against nonzero state and it fails, off by 2.35e-6 against a 1e-7 tolerance.
Each link below shares sources, entities, or timing with this story.
The file is 1.56TB. That's the first thing you notice about the moonshotai/Kimi-K3 Hugging Face repo that went live today. 2.8 trillion total parameters, 104B activated, 896 routed experts with 16 selected plus 2 shared per token, 93 layers split 69 Kimi Delta Attention and 24...
The minute Fable 5 and Mythos 5 went dark for foreign nationals, r/LocalLLaMA found its answer. Moonshot AI's Kimi K2.7 Code is a 1T-parameter MoE (32B active, 384 experts), 256K context, shipped under a Modified MIT license. The headline number that's getting it pulled: 81.1...
Cursor shipped Composer 2 on March 19, marketing it as a proprietary in-house model. Within 24 hours, a developer found the API routing to kimi-k2p5-rl-0317-s515-fast. The model powering the most-hyped coding tool update of the month was Moonshot AI's Kimi K2.5 with continued...
Moonshot AI replaces standard fixed residual connections with softmax attention over preceding layer outputs. Already in production at 48B scale (Kimi Linear). Consistent scaling improvement validated across model sizes. 1,330 HuggingFace upvotes. Source
Moonshot AI released Kimi K3, a sparse mixture-of-experts activating 16 of 896 experts per token. That's about 1.8% of the pool live at any moment, with a 1M-token context window and native vision. Two new architectural pieces show up: Kimi Delta Attention and Attention Residu...
Moonshot exposes an Anthropic-compatible endpoint, so pointing Claude Code at K3 means setting the Anthropic base URL and supplying a Moonshot key. No new CLI, no config rewrite. Hosted at $3/$15 per Mtok, same tier as Claude Sonnet 4.6, and Artificial Analysis scores K3 at 57...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.