From Layers to Submodules: Finer-Grained LLM Compression via Replacement at Submodule Level

arXiv·medium signal

Challenges the standard approach of replacing entire transformer layers during LLM compression, showing that submodule-level replacement (attention heads, FFN blocks) achieves better accuracy-compression tradeoffs. Existing replacement methods use full-layer granularity plus a single fitting strategy; this work relaxes both constraints. For teams deploying compressed models in production, submodule-level pruning could recover quality lost by layer-level approaches.