APEX publishes a verification-first LLM inference tile in RTL with the KV-cache codec inside the datapath, 0.56 tok/s measured on FPGA

GitHub·medium signalXBlueskyLinkedInCopy link

SigmanticAI/apex-inference-chip landed 2026-08-17 (Apache-2.0, 675 stars, 1 fork, pushed 2026-08-18). It is one transformer decoder layer in real RTL, attention, softmax, RMSNorm, RoPE, SwiGLU and residual, each block bit-exact against an executable golden model, with real Qwen2.5-0.5B run through the verified pipeline and brought up on FPGA at a measured 0.56 tok/s after a 140x climb. The architectural bet is putting the KV codec in the datapath so read speed stays flat as context grows, rather than quantizing the cache in software. The repo labels every number as measured or projected, which is rarer than it should be in accelerator repos.