Sources
Needle2 Is a 14 MB, 45M-Parameter Agentic Model That Runs on an ESP32-S3 and Does 500+ tokens/sec Decode on a Raspberry Pi 5
Cactus Compute released Needle2 on Aug 10 under Apache 2.0: 45M parameters, 14 MB on disk, 28 MB peak session RAM, and 70 MFLOPs per token via CQ2-bit compression. Measured throughput is 800+ tokens/sec prefill and 500+ decode on a Raspberry Pi 5, 400–1,500 tokens/sec on Quest 3S and Apple Vision Pro, and 300–700 on budget Samsung A-series phones, with targets spanning Cortex-M microcontrollers through x86 and WebAssembly. It is not a general chat model — on the 961-row Mobile Actions benchmark it gets 98.3% function-name accuracy but 71.3% single-call and only 48.4% two-call accuracy, so the honest use case is single-shot tool dispatch on hardware where nothing else fits, not multi-step reasoning.
↳ Follow the thread