Web search agents don't get better the more they search

Researchers ran the first end-to-end comparison of agentic web search across ChatGPT, Claude, Grok and DeepSeek, mixing real user interactions with controlled API tests on the same models, according to the paper. How often each platform decided to search varied a lot, and searching more didn't produce better responses.

That matters for anyone treating search-enabled answers as more trustworthy by default. If invocation rate doesn't track quality, then a chatbot that searches five times isn't necessarily more reliable than one that searches once, and users have no way to tell from the outside which situation they're in.

Two more findings compound the problem. Each platform's search results skew toward a different set of preferred domains, so the same question can surface different sources depending on which assistant you ask. And while most responses were grounded in what got retrieved, the paper found some claims rested on sources that were never cited.

The paper doesn't say which domains each platform favors or how large the uncited-claim problem is in absolute terms, so it's not clear yet whether this is a rare failure mode or a routine one. Anyone building on top of these search tools should treat cited-and-grounded as a claim to verify per response, not a property of the platform.

Worth watching is whether any of the four platforms responds by publishing their domain weighting or citation coverage numbers. Right now that's a black box on all four.