NeuronMoE: Neuron-Guided Mixture-of-Experts for Efficient Multilingual LLM Extension

arXiv 2603.05046·medium signal

Analyzes language-specific neurons across all transformer components to guide per-layer expert allocation in MoE architectures, enabling targeted expert specialization by language without full-scale pretraining. Achieves approximately 40% average parameter reduction while maintaining baseline multilingual benchmark performance. Relevant as MoE becomes the dominant architecture for production-scale LLM deployments and multilingual coverage becomes a key differentiator.