MoonshotAI open sources MoonEP for skewed MoE training

MoonshotAI released MoonEP on July 24, an open source library for mixture-of-experts training, under the MIT license. It keeps expert-parallel communication time flat even as the router sends wildly uneven token loads across experts, a pattern the field calls skew.

Router skew breaks most MoE training setups. Tokens flood some experts while others sit idle, and that imbalance chokes the communication step the whole system depends on. MoonEP holds every one of its 8 expert-parallel ranks to exactly S × K tokens, no matter how skewed the routing gets.

The library reads the router's live output and plans a small set of redundant experts on the fly. It prefetches those experts, then reduces their gradients back to the home rank only in the backward pass.

Tokens get written straight into expert-grouped positions on remote ranks, a zero-copy path that skips the permute-in and permute-out steps most MoE pipelines need. Fixed, static shapes also remove the per-layer host synchronization that slows other systems down.

Benchmarked against DeepEP v2 on H20 GPUs with those 8 expert-parallel ranks, communication time stays close to flat as skew climbs, per the project's repo. DeepEP v2 degrades steadily over the same range, and under high imbalance it runs out of memory, a failure mode MoonEP is built to avoid.

That's a specific bet: MoE training's bottleneck wasn't clever scheduling, it was uneven token counts. Whether it holds outside Moonshot's own numbers depends on what happens past 8 ranks, the one variable this benchmark doesn't test.