Evaluation Context Protocol proposes a JSON-RPC contract layer for agent evals across frameworks

arXiv·medium signalXBlueskyLinkedInCopy link

ECP (arXiv 2608.19263, August 18) responds to benchmark gaming and fragmented eval tooling with a portable evaluation contract exposed over JSON-RPC, capturing agent outputs, tool invocations and audit context uniformly. Adapters ship for LangChain, LlamaIndex, CrewAI and PydanticAI so the same programmatic checks run against any of them. The authors label it work-in-progress and say the evaluation surface, method set and grader families are all expected to change, so it is worth watching rather than adopting.