Pythia: Predictability-Driven Agent-Native LLM Serving for Multi-Agent Architectures

arXiv·high signal

Introduces a serving infrastructure designed specifically for multi-agent LLM workloads, exploiting the structural constraints of agent workflows (fixed tool schemas, predictable turn-taking, constrained output formats) to optimize scheduling and resource allocation. Unlike generic LLM serving that treats each request independently, Pythia leverages semantic information about agent roles and workflow DAGs to predict compute needs and batch intelligently. Directly relevant to anyone running multi-agent systems at scale — addresses the gap between single-model serving and real agent orchestration.