A 3.16M-parameter LLM hits 21,300 tok/s on a $250 FPGA by never touching DDR.

Mike Ayles' August 10 writeup runs an INT4 transformer, about 1.5 MB of weights trained on TinyStories, entirely in on-chip SRAM on a Xilinx Kria KV260 with zero DDR in the token-generation loop. Peak aggregate throughput is 59,965 tok/s at 200 MHz, with the live demo sustaining ~21,300 tok/s under 2,000 concurrent connections. Same board's ARM A53 cores manage 11 tok/s; a laptop RTX 3050 Ti manages 719. The entire result is trading 20 GB/s shared DDR for hundreds of GB/s of on-chip bandwidth.