ACE Skips MoE Experts With No Training, No Calibration Data, and No Checkpoint Changes

arXiv 2609.05228·medium signalXBlueskyLinkedInCopy link

arXiv 2609.05228 targets the redundancy in fixed top-k MoE routing, which activates the same number of expert slots for every token. ACE combines a Global Spectral Proxy estimating transformation capacity from the coupled gate, up, and down projections with RMSNorm scaling, and Router-Conditioned Refinement building expert-specific direction prototypes from centered router weights. An expert slot is skipped only when both views call it low-contribution, and the top-1 expert is always retained; all expert statistics are precomputed offline so inference adds only table lookups and scalar operations, evaluated across three MoE LLMs and eight benchmarks.