Two Independent Sources on an FPGA KV Fabric: 58% Less Interconnect Traffic at 8K-32K Context

arXiv 2609.19207·low signalXBlueskyLinkedInCopy link

arXiv 2609.19207 (submitted 16 Sep 2026) treats KV cache movement, not compute, as the binding constraint on tiled decoding accelerators, arguing prior compression and DRAM-placement work still funnels traffic through centralized memory paths. MeshKV moves KV blocks as packetized flows over a lightweight network-on-chip, combining affine striping to spread block homes, multicast with verified duplicate suppression, and a stage that overlaps prefetch, tile multiply and streaming softmax behind credit-aligned FIFOs. On an 8x8 FPGA implementation with LLaMA-2-7B and Mistral-7B at 8K-32K context it cuts interconnect traffic up to 58%, improves KV bandwidth utilization 2.1x, and delivers up to 1.9x multi-stream throughput.