Embedding retrieval gains flatten out at 768 dimensions

Retrieval quality flattens past 768 embedding dimensions, according to CodeSOTA's MTEB benchmark analysis. That's the tradeoff anyone re-indexing a vector database needs to see before scaling up embedding size. Storage grows with dimension count, and past 768 dimensions, most of that spend isn't buying better search results.

The jump from 256 to 768 dimensions buys recall gains worth the storage cost. Push further, from 1536 to 3072, and the gains turn marginal, while storage costs roughly 6x more.

The analysis also argues for hybrid retrieval over dense embeddings alone. BM25, added alongside dense retrieval, still wins on exact matches: IDs, product names, and rare entities.

CodeSOTA flags its own analysis as single-source and secondary, and says to verify against your own eval set before re-indexing anything. That's a fair caveat. Benchmark aggregates don't know your data distribution or your query patterns.

Teams still defaulting to 3072-dimension embeddings without a BM25 layer are spending storage on recall gains a hybrid index would get them for free. If you're planning a re-index, 768 dimensions plus BM25 is the more defensible default than reaching for the biggest embedding model available.