Pruning MoE Experts Independently Is a First-Order Approximation, and Fixing That Adds 6.1% on Agentic Coding

arXiv 2609.18916·medium signalXBlueskyLinkedInCopy link

arXiv 2609.18916 (16 Sep 2026) points out that existing expert-pruning methods decide each expert in isolation and assume contributions are purely additive, when MoE expert usage is cooperative. HOPE derives a second-order objective that provably minimizes an upper bound on pruning error, and shows the state-of-the-art first-order method REAP is the special case where interaction terms are dropped. Across three frontier MoE models up to 122B parameters, two calibration sets, and math, instruction-following, coding and agentic benchmarks, HOPE averages rank 1.58 of 5 methods at 50% pruning versus 2.42 for REAP, with gains up to +6.1% on agentic coding and the widest margin at high pruning rates.