Give each agent role its own LoRA expert instead of just a different prompt

arXiv 2608.09251·medium signalXBlueskyLinkedInCopy link

MoRSE argues that prompt-level role differentiation is the ceiling on current multi-agent systems and adds parameter-level specialization: subtasks are laid out as a dependency-aware DAG, and a dynamic mixture of (role, subtask) LoRA experts with semantic routing adapts a single shared base model per agent. Training uses hierarchical group-relative policy optimization with two-layer credit assignment that separates expert quality from routing decisions. It improves both full-task and incremental-step performance on code-generation benchmarks and generalizes to unseen task categories — a concrete path for teams that have hit the limit of prompt-only agent roles without paying for N separate models.