A typed knowledge graph over 690 agent skills performs 11.2 points WORSE than a plain hybrid ranker
A study submitted 2026-08-06 (arXiv 2608.06196) compares two ways to pick skills from a 690-skill library: a hybrid lexical+dense ranker versus a typed knowledge graph encoding prerequisites, data flow and ordering. On 117 realistic non-echoing queries the ranker hits top-5 in 73.5% ±8.0 of cases, and substituting graph neighbours for ranked results at matched token budget is significantly worse (-11.2 points, p=0.0007). The mechanism is a pre-filter topology bound — 98.6% of typed edges connect skills the ranker had already surfaced together, and 73% of the queries the ranker misses are unreachable through the graph at all. They also show evaluating on author-written queries overstates hit@5 by up to 44 points, which would have hidden the whole result. Directly relevant to anyone building skill-marketplace routing on graph structure.
Source
↳ Follow the thread