UniPool: Single Shared Expert Pool Matches MoE Performance with 41-67% Fewer Expert Parameters

arXiv·medium signal

UniPool replaces per-layer expert sets with a globally shared pool accessed through independent routers. Across five model scales (182M-978M), reduced-pool variants using only 41.6-66.7% of standard expert parameters match or exceed vanilla MoE performance. Replacing a deeper layer's learned router with uniform random routing drops accuracy by only 1.0-1.6 points, revealing massive redundancy in conventional per-layer designs.