Adding irrelevant items to RAG prompts raised relevance accuracy

A controlled study found that mixing irrelevant items into a fixed-length prompt beat an all-relevant one at judging relevance, per a new study posted to arXiv.

That's backwards from how I've built every retrieval setup I've shipped: rank by similarity, keep the top hits, drop the rest on the assumption that noise hurts accuracy. The study ran 2,420 trials across 11 model configurations on 661 real workplace items, holding the prompt at ten slots. Half the trials filled all ten slots with high-relevance items; the other half swapped five of those ten for same-domain items the model should have scored as noise. The 50:50 mix beat the all-relevant prompt on relevance-score concordance by 0.077, a moderate effect (Cohen's d = 0.49, Holm-adjusted p < .001, n = 220).

The authors' explanation is discrimination, not clutter. Ten relevant items give a model nothing to compare against, so everything reads as important. Add same-domain distractors and the model gets a floor to score from. That's a narrow claim, though: same-domain low-relevance items, not random junk from an unrelated corpus, and it held for scoring, not for generation.

The same paper's engineering section is the more provable part: fetch-once/process-locally and five other production patterns cut cold-load latency from a 3.5-10.5 minute baseline down to 61-116 seconds, with an estimated 60-70 percent fewer tokens. None of that is new to anyone who has built agent infrastructure. The scoring result is.

The bet

If this replicates outside one team's task, every aggressive similarity threshold engineers have tuned for precision is quietly suppressing accuracy it could have kept. The move to test: take your current top-k, swap the bottom half for same-domain items you'd normally throw out, and measure the relevance score, not the final answer. I wouldn't touch a reranker over one study, but I'd run that A/B before changing anything else.