LangChain's benchmark harness: keep a frozen lite subset, 8x faster and 6x cheaper.

Their July 23 post on benchmarking Deep Agents gives a three-part harness: Harbor-Index (82 tasks distilled from 6,000+ candidates across 54 benchmarks), τ³-bench (30 multi-turn tasks with simulated users), and ContextBench (30 retrieval tasks) (LangChain). Two practices transfer to anyone tuning an agent: maintain a frozen lite subset weighted toward hard-but-solvable tasks for iteration before committing to a full run, and keep fast deterministic unit tests alongside the benchmarks each asserting one specific harness behavior. They stress running every task multiple times, since single-sample agent scores are noise. I learned that one the expensive way.