Mixture of Roles Matches Multi-Agent Performance in One Turn at 20x Lower Token Cost

arXiv 2608.27338·medium signalXBlueskyLinkedInCopy link

MoRe learns a codebook of steering vectors, each encoding a latent role, and uses a query-aware router to fuse them into a single composed steering vector for single-turn inference, giving multi-perspective specialization without multi-turn agent orchestration. The backbone LLM stays frozen and training runs as a three-stage SFT curriculum plus GRPO post-training. Across reasoning and personality benchmarks it beats single-agent baselines by 2.2% on average and matches multi-agent systems while cutting token cost 20x, which is a direct counter-argument to defaulting to a multi-agent pipeline for role diversity.