DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization Cuts Edge MoE Inference I/O Overhead

arXiv·medium signalXBlueskyLinkedInCopy link

DyMoE proposes a combined dynamic expert selection and mixed-precision quantization approach that significantly reduces memory footprint and I/O overhead for MoE model inference on edge devices. The key insight is that most experts are dormant for any given token — DyMoE skips loading them entirely and quantizes remaining experts at runtime based on activation confidence. This is directly relevant to on-device agent deployments (Samsung Galaxy-class, Copilot SDK + Foundry Local scenarios) where MoE models are otherwise impractical due to full-weight I/O cost.