CrewAI, AutoGen score 65% privacy risk in red-team test

Red teams testing CrewAI and AutoGen scored 65% privacy risk in multi-agent configurations, per a paper posted to arXiv.

That number matters for anyone wiring agents together to move data between them. Standard chat-safety testing doesn't check what happens when one agent's output becomes another agent's input.

The test needed only a basic system description to run. It draws on a seven-domain risk taxonomy, generating 120 automated adversarial scenarios per domain, then scoring them with an LLM judge validated by humans. Across four base models, average governance risk came in at 56.25%. Agent-behavior vulnerability, meaning the agents acting on adversarial prompts rather than just leaking data passively, reached 85%.

A single chatbot has one input and one output, and existing red-team tools already cover it. Add a second agent and a data path opens between them that single-turn chat evaluation never touches. Information one agent shouldn't disclose can move to a second agent that has no reason to withhold it, since nobody designed a boundary between them.

The paper doesn't break out which of the seven risk domains drove the governance score versus the behavior score. It also doesn't say whether either framework has since patched specific scenarios, or whether the 65% figure holds across newer model versions.