WIDE Prunes LLM Width Per Token, Delivering 4.95x Decoding and 1.98x Prefill Kernel Speedups at 50% Sparsity

arXiv·high signalXBlueskyLinkedInCopy link

WIDE (arXiv 2607.28418, July 30) is an end-to-end differentiable token-level dynamic width pruning framework for both prefill and decode, letting each token select its own attention-head groups and FFN-channel groups — pushing dynamic sparsity from layer-level down to neuron-block granularity. A pruning/kernel co-design splits acceleration into mask reordering, hardware-agnostic block-level skipping, and hardware-dependent intra-block skipping. At 50% sparsity it reports a 55.1% quality boost over state-of-the-art dynamic depth pruning under calibration-only settings, with kernel speedups up to 1.98x prefill and 4.95x decoding and 1.68x/1.55x end-to-end; code is released.