PagedWeight: Dynamic Quality-Aware Weight Quantization Cuts MoE Serving Memory Under KV-Cache Pressure

arXiv·high signal

A new arXiv paper (2607.16184) from Yuchen Yang, Yifan Zhao and Anisha Dasgupta targets the specific failure mode where Mixture-of-Experts models — normally the efficiency win — become memory-bound in KV-cache-intensive serving, since expert weights and cache compete for the same HBM. PagedWeight applies quality-aware quantization dynamically per expert page rather than a uniform static bit-width across the whole model. For builders self-hosting MoE models like Qwen or DeepSeek on constrained GPUs, this is the class of technique that decides whether long-context serving fits on one card or two.