Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-23 · high
A two-model setup strips retrieved chunks down to the exact spans a query needs before the answering model ever sees them, per AWS.
Why now: AWS published the benchmark on August 21.
AWS cut RAG token volume 8.6x by filtering retrieved chunks down to only the spans a query needs, per an August 21 benchmark. That's a 33% cost cut while holding 97.5% of baseline answer quality, using Bedrock's own composite score. For anyone running retrieval pipelines at volume, that's real money back without a quality hit big enough to matter.
The setup uses two calls. Claude Haiku reads each retrieved chunk at temperature 0.0 and extracts the exact query-relevant text verbatim, no summarizing. Claude Sonnet then answers using only those extracted spans instead of the full chunk set.
Add a reranker before the extraction step and the numbers improve again. 10.1x fewer tokens, 36% savings, at 12% higher latency.
The hallucination numbers matter more than the token math. Compression alone cuts hallucination rate 7 points. Reranker plus compression cuts it 13 points, per AWS's numbers. Feeding the answering model less text, but the exact right text, seems to make it more accurate. Cheaper too.
Verbatim extraction is doing the work here, not summarization, per AWS's writeup. A summarizing extractor could still inject its own phrasing or drop context in ways that are hard to check. Pulling exact spans keeps the chain traceable. You can point at the sentence in the source chunk that produced the answer.
If you're running Bedrock RAG at full-chunk context on every call, this is a cheap benchmark to replicate against your own eval set. Try it before reaching for a bigger context window.
Each link below shares sources, entities, or timing with this story.
AWS uses Claude Sonnet / Shared entities / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (AWS uses Claude Sonnet); both cover AWS, RAG; reported by the same outlet (aws.amazon.com).
AWS uses Claude Sonnet / Shared entities / Same source domain / What happened next / Tension
Linked by a graph relationship (AWS uses Claude Sonnet); both cover August, AWS; reported by the same outlet (aws.amazon.com).
AWS uses Claude Sonnet / Shared entity: AWS / Same source domain / Shared topic / Earlier coverage
Linked by a graph relationship (AWS uses Claude Sonnet); both cover AWS; reported by the same outlet (aws.amazon.com).
AWS uses Claude Sonnet / Shared entities / Same source domain / Earlier coverage
Linked by a graph relationship (AWS uses Claude Sonnet); both cover August, AWS; reported by the same outlet (aws.amazon.com).
AWS uses Claude Sonnet / Shared entities / Shared topic / Earlier coverage
Linked by a graph relationship (AWS uses Claude Sonnet); both cover August, AWS; overlapping topics (claude, token).
AWS uses Claude Sonnet / Shared entities / Same source domain / Earlier coverage
Linked by a graph relationship (AWS uses Claude Sonnet); both cover August, AWS; reported by the same outlet (aws.amazon.com).
Linked by a graph relationship (AWS uses Claude Sonnet); both cover August, AWS; reported by the same outlet (aws.amazon.com).
Linked by a graph relationship (AWS uses Claude Sonnet); both cover August, AWS; reported by the same outlet (aws.amazon.com).