Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-25 · high
Protecting a single exponent bit pushes the threshold past 79,000 flips across five more open models.
Why now: Nobody's tested this on closed-weight models yet, so anyone running inference on consumer or edge hardware has no way to know where their own failure threshold sits.
A simulated cosmic-ray strike breaks an LLM after about 20 random bit flips, per a bit-flip simulation posted at spock.is.
The failure isn't spread evenly across the weights. Testing Qwen2.5-Coder-3B in FP16, the fatal flip almost always struck bit 14, the most significant bit of the floating-point exponent. Protecting that one bit pushed the model's resilience from about 23 flips to somewhere between 79,000 and 490,000, depending on the random seed.
The same roughly-23-flip threshold held across five more models: Mistral-7B, Qwen3-8B, phi-4, Granite-4-8B and Qwen3-14B. That consistency points at the float format itself, not model size, as the source of the weakness.
Quantization changed the math. Q4_K_M weights survived a median of 1,024 random flips before failing, about 49 times more than FP16's median of 22. A format built to save memory turned out to also absorb damage that would kill a full-precision model outright.
The simulation doesn't say whether ECC memory or other hardware error correction would catch these flips before they reach inference, so it's not clear how much of this risk survives outside a lab. For anyone running inference on consumer hardware, or anywhere radiation exposure runs higher than a data center, that gap matters.
Each link below shares sources, entities, or timing with this story.
Simon Willison released LLM / Shared entity: Coder / Shared topic / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover Coder; overlapping topics (between, model).
Simon Willison released LLM / Shared entity: LLM / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-08-16.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-07-27.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-19.
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-06-18.
Simon Willison released LLM / Shared entity: Qwen3 / Earlier coverage / Tension
Linked by a graph relationship (Simon Willison released LLM); both cover Qwen3; earlier Qwen3 coverage from 2026-04-23.
LLM uses OpenAI / Shared entity: LLM / Earlier coverage
Linked by a graph relationship (LLM uses OpenAI); both cover LLM; earlier LLM coverage from 2026-08-21.
Simon Willison released LLM / Shared entity: LLM / Earlier coverage
Linked by a graph relationship (Simon Willison released LLM); both cover LLM; earlier LLM coverage from 2026-08-17.