Liquid AI Ships Pipette, an Open Benchmark Stack Measuring 35 Model Classes and 7 Quantizations on Real Phones
AINews on 2026-08-25 reports Liquid AI's Pipette suite for on-device inference, covering quality, speed, latency, and memory across model, quantization, runtime, and device combinations, with over 10,000 verified results spanning 35 model classes, 7 quants, llama.cpp runtimes, and four devices. The GitHub org backs this up: Liquid4All/pipette-clients holds iOS and Android measurement harnesses and Liquid4All/pipette-scores holds a stateless model-blind scoring service, both pushed within the last week. Under an 8 GB memory and 16K context framing, Nanbeige4.2-3B and LFM2.5-2.6B tie at 63 average, but LFM2.5-2.6B answers in 8.0s at 2.3 GB on iPhone against Nanbeige's 21.4s at 4.0 GB, and MoE designs activating around 1B parameters per token get under six seconds.
↳ Follow the thread