VI-MoLE Argues Uncertainty Is the Wrong Trigger for Activating More LoRA Experts

arXiv 2608.02528·low signalXBlueskyLinkedInCopy link

Dynamic routers for mixtures of LoRA adapters typically activate more experts when the router or prediction is uncertain, silently equating uncertainty with useful extra computation — but an uncertain example may stay ambiguous even after every expert agrees. VI-MoLE reframes routing as certified value-of-information allocation: it learns the counterfactual risk remaining after each expert prefix, converts predictions into simultaneous upper-risk certificates on held-out calibration data, and spends a global adapter budget on the token-layer action with the largest certified marginal risk reduction per unit cost, with a terminal certificate deciding answer-versus-abstain. The paper proves simultaneous certificate validity, optimal greedy allocation under diminishing certified gains, and allocation regret under value-estimation error.