Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-04 · high
Critics in the thread say the real bottleneck is prefill: about 30 minutes to process 10,000 tokens on an M5.
Why now: The demo is sitting at 215 points on Show HN as of the Aug 4 roundup, with the prefill objection still topping the comments.
Swiftlet runs an 80-billion-parameter Qwen model inside 4.3 GB of RAM by streaming its mixture-of-experts weights straight off the SSD, according to its Show HN post.
That's small enough for a consumer laptop, not a server rack. On an M5 Mac, the setup manages 4.5 to 5 tokens per second, and a 35B version runs on an iPhone.
Commenters didn't let the headline number stand. Prefill, the step where the model reads the prompt before generating, is the real cost. It takes roughly half an hour for 10,000 tokens on an M5, per critics in the thread.
Generation is fast once it starts. Getting there isn't.
The trick only works because Qwen is a mixture-of-experts model that activates about 3B of its 80B parameters for any given token. Swiftlet has to pull those active experts off disk, not the full model. Dense models don't get that shortcut, since every parameter fires on every token regardless of size.
Disk-streaming inference only works because Qwen activates about 3B of its 80B parameters per token. That's why the trick scales with MoE adoption, not with local AI in general. Watch whether more labs ship MoE-first architectures, because that's what decides if this stays a demo or becomes a real way to run big models on small hardware.
The demo is sitting at 215 points on Show HN as of the Aug 4 roundup, with the prefill objection still topping the comments.
Each link below shares sources, entities, or timing with this story.
Swiftlet uses Qwen / Shared entities / Shared topic / Earlier coverage / Tension
Linked by a graph relationship (Swiftlet uses Qwen); both cover MoE, Qwen; overlapping topics (dense, model, token).
Linked by a graph relationship (Swiftlet uses Qwen); both cover MoE, Qwen; overlapping topics (architecture, model).
Swiftlet uses Qwen / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (Swiftlet uses Qwen); both cover MoE, Qwen; overlapping topics (hour, model, token).
Linked by a graph relationship (Swiftlet uses Qwen); both cover MoE, Qwen; overlapping topics (active, dense, model).
Linked by a graph relationship (Swiftlet uses Qwen); both cover MoE, Qwen; overlapping topics (dense, model).
Alibaba released Qwen / Shared entity: MoE / Shared topic / Earlier coverage / Tension
Linked by a graph relationship (Alibaba released Qwen); both cover MoE; overlapping topics (active, model).
Swiftlet uses Qwen / Shared entities / Earlier coverage
Linked by a graph relationship (Swiftlet uses Qwen); both cover MoE, Qwen; earlier MoE coverage from 2026-06-27.
Alibaba released Qwen / Shared entity: MoE / Shared topic / Earlier coverage
Linked by a graph relationship (Alibaba released Qwen); both cover MoE; overlapping topics (hour, model).