WIDE prunes model width per token for 4.95x decoding speedup.

WIDE, a new pruning method, cuts decoding time by up to 4.95x by letting each token choose its own attention-head and FFN-channel groups, per the paper posted to arXiv.

That's the tradeoff pitch: at 50% sparsity, WIDE beats the best existing dynamic depth pruning method by 55.1% on quality, using calibration data alone. No retraining pass is required.

Most pruning approaches decide sparsity at the layer level, dropping the same blocks for every token that passes through. WIDE pushes that decision down to individual attention-head groups and FFN-channel groups, and it runs across both the prefill and decode passes rather than one. Each token ends up with its own pruning pattern, set by an end-to-end differentiable framework trained into the model rather than picked by hand.

The kernel numbers back that up. Prefill runs up to 1.98x faster and decoding up to 4.95x faster, with 1.68x and 1.55x gains once measured end-to-end. The authors released their code alongside the paper, so anyone can check the speedup claims against a real implementation instead of a PDF table.

The paper doesn't specify which base models it tested WIDE on, so it's unclear how the speedups translate across architectures.