Two-stage retrieval with Matryoshka embeddings: truncated vectors for candidates, full vectors to re-rank

Towards Data Science·medium signal

Matryoshka Representation Learning packs usable embeddings at multiple sizes into one vector, so you can truncate a 1024-d embedding to 256/128/64-d with minimal recall loss — up to 16× less storage going 1024→64. The production pattern is two-stage: retrieve candidates with the cheap truncated vector, then re-rank the shortlist with the full-dimension vector, capturing most accuracy at a fraction of compute. Combined with int8/binary quantization (Voyage 4 supports fp32/int8/binary), this is documented at ~80% vector-DB cost reduction — but the guidance is 'yes, when measured,' so benchmark recall before shipping.