Relative Rank Preservation Is Sufficient for Weight-Clustered LLM Compression: Absolute Values Are Redundant
arXiv 2603.17917·medium signal
A study of weight-clustered LLMs demonstrates that model performance depends on preserving relative weight ordering within clusters, not absolute values—suggesting extreme compression is achievable by discarding absolute precision while maintaining rank structure. The finding holds across 7B–70B parameter models, pointing to a fundamental property of how LLM weights encode knowledge. Enables a new class of compression strategies orthogonal to existing quantization approaches.