AWS proposes task-aware knowledge compression as a RAG replacement at 1.6% of baseline token cost

AWS Machine Learning Blog·medium signalXBlueskyLinkedInCopy link

An AWS Machine Learning Blog post published July 27 describes task-aware knowledge compression (TAKC), which pre-compresses an entire knowledge base into task-specific representations rather than retrieving chunks at query time — different tasks get different compressions of the same source document. Four tiers are offered: 8x (~87.5% context reduction), 16x, 32x, and 64x (~98.4% reduction), with a 100K-token knowledge base at 1,000 queries/day costing 6.25%, 3.1%, and 1.6% of baseline versus 10% for top-10-chunk RAG. The reference architecture is two serverless pipelines on Lambda, Bedrock, ElastiCache Serverless, S3, API Gateway, and Cognito, with compression via Claude 3 Haiku, Claude 3 Sonnet, or Amazon Titan Text.