Cut vector-store cost with quantization plus Matryoshka embeddings

Towards Data Science·medium signal

Two orthogonal levers cut vector-store cost without much recall loss: quantization (float32→int8 scalar for ~4x storage/latency savings with minimal quality drop; binary for 32x but steep recall loss) and Matryoshka (MRL) embeddings that let you truncate dimensions on demand for up to ~80% cost reduction. Combine them—store binary/int8 for the ANN first pass, then rescore survivors with full-precision vectors. Reach for this when embedding storage or P95 latency, not retrieval quality, is the binding constraint.