iPhone 17 Pro Demonstrated Running a 400B Parameter LLM On-Device via SSD-to-GPU Weight Streaming

Hacker Newsvia @anemll·high signal

Developer @anemll demonstrated an iPhone 17 Pro running Qwen3.5-397B-A17B (2-bit quantized, 17B active params) entirely on-device using Flash-MoE SSD-to-GPU streaming. The MoE architecture activates only 4-10 of 512 experts per layer, avoiding loading all 400B parameters into memory. Performance is 0.6 tokens/second — impractical for daily use but a proof-of-concept that mobile hardware can run frontier-scale models.