Semantic chunking: split documents where consecutive-sentence cosine similarity drops below a threshold
MyEngineeringPath·medium signal
Instead of fixed-size windows, embed sentences and start a new chunk each time cosine similarity between consecutive sentences falls below a set threshold, so chunk boundaries fall on real topic shifts. This keeps semantically coherent context together and is a cheap preprocessing change that improves retrieval precision before any reranking is added.