An FPGA KV fabric cuts interconnect traffic up to 58% at 8K-32K context.

arXiv 2609.19207 treats KV movement rather than compute as the binding constraint on tiled decoding accelerators, arguing prior compression and DRAM-placement work still funnels traffic through centralized memory paths. MeshKV moves KV blocks as packetized flows over a lightweight network-on-chip, with affine striping to spread block homes, multicast with verified duplicate suppression, and a stage overlapping prefetch, tile multiply and streaming softmax behind credit-aligned FIFOs. On an 8x8 FPGA with LLaMA-2-7B and Mistral-7B it improves KV bandwidth utilization 2.1x and delivers up to 1.9x multi-stream throughput. arXiv