Similarity-Aware Mixture-of-Experts for Data-Efficient Continual Learning
arXiv·medium signal
McLaughlin et al. introduce a MoE architecture that uses similarity-aware expert routing to enable data-efficient continual learning without catastrophic forgetting. The approach dynamically activates experts based on input similarity to previously learned distributions, requiring significantly less data per new task. Relevant for production ML systems that must adapt to distribution shifts post-deployment without full retraining.