Chunk Coverage: A Structural Test Metric for RAG Retrieval, 1.7× Faster to 50% Coverage Than Random

arXiv 2607.18155·medium signal

RAG test suites usually evaluate end-to-end answers, leaving the retrieval component untested as a structural surface. Chunk Coverage measures the fraction of corpus chunks retrieved at least once across a test suite, exposing which parts of the retrieval space were never exercised. Evaluated on clinical and financial RAG scenarios, CC-guided test selection reached 50% coverage 1.7× faster than random and 4.2× faster than redundancy-biased strategies, and improved fault-detection effectiveness (APFD) by 10–25% over random. It gives RAG builders a code-coverage-style target for the retriever, not just the generator.