Survey of Open-Source Multi-Agent Frameworks Finds No ROUGE Difference Between Them — and Telemetry Missing Across the Board

arXiv 2608.11965·medium signalXBlueskyLinkedInCopy link

This mixed-method experience report first quantitatively analyzes the most relevant open-source multi-agent-system frameworks on documentation, features, and capabilities from a developer's perspective, then qualitatively implements one common use case — README summarization — across a subset of them. Frameworks cover the fundamental MAS components well, but advanced capabilities such as agent telemetry are still largely absent. The empirical result is deflating and useful: there was no significant difference in ROUGE scores across frameworks on the summarization task, suggesting framework choice should be driven by observability and coordination ergonomics rather than expected output quality.