Hacker News
NEEDLE Is a Continuously Refreshed Search Benchmark Built From Real Agent Search Logs
Published August 27 by Keenable, NEEDLE (News, Everyday, Finance, Deep-tail, Legal Evaluation) rotates queries continuously to defeat overfitting and leakage, drawing from real agent search logs plus algorithmically generated production-shaped queries. Categories include hourly-refreshed News from RSS and Google Trends, Finance from Wikidata and GLEIF filings, Scholar, AgenticRare long-tail entities, and Legal from court opinions and the CFR, tested against Google via Serper, Bing via SearchAPI, Brave, Tavily, Parallel and Exa. No numeric leaderboard is published and the vendor ranks itself alongside Google, so treat the claim as unverified; the useful part is the construction method for anyone evaluating agent search.
↳ Follow the thread