PROTEA: First Offline Evaluation and Iterative Refinement Framework for Multi-Agent LLM Workflows

arXiv·medium signalXBlueskyLinkedInCopy link

Multi-agent LLM workflows outperform single-prompt baselines but are notoriously difficult to debug — failures originate from subtle errors in intermediate outputs that propagate across agents. PROTEA provides an offline evaluation framework that traces failures back to specific agent interactions and enables iterative refinement without expensive live runs. Practical for anyone building multi-agent orchestration pipelines who needs to debug agent-to-agent communication failures.