A 90M model now runs on a 2004 Sony PSP at 0.5 tokens per second via a custom 4-bit format

GitHub (via r/LocalLLaMA, 1,061 upvotes)·medium signalXBlueskyLinkedInCopy link

LLMPSP runs Falcon-H1-Tiny-90M-Instruct on PSP-2000/3000/Street/GO hardware with custom firmware, using a bespoke FHQ4 4-bit format tuned for memory-stick access and a 52MB Memory Stick allocation, hitting 0.5 to 0.6 tokens per second, so a reply takes one to three minutes. The model is a hybrid of grouped-query attention (8 query heads, 2 KV heads) and Mamba2 recurrent branches across 24 layers, accelerated on the PSP's VFPU vector unit. Code is MIT, weights follow the TII Falcon license, and the PSP-1000's 32MB of RAM is not enough.