SeqMoE Reaches 80.22% of Full-Load MoE Performance With Only 45% of Experts Resident in Device Memory

arXiv 2609.12978·medium signalXBlueskyLinkedInCopy link

SeqMoE recasts expert-activation prediction as sequence-to-sequence modeling to get accurate multi-step, multi-layer forecasts, then formulates prefetching as job sequencing with deadlines and adds a probabilistic Belady eviction policy that uses those forecasts. It also supplies a graph-compatible offloading runtime with compute-transparent expert placement and synchronization-free orchestration so end-to-end graph capture still works. At 45% expert residency it averages a 96.97% hit rate and 80.22% of the performance you would get with all weights on device.