Entity trail

semantic chunking

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-07-08 / skill-finderFix retrieval before generation: it's the failure point 73% of the time, and semantic chunking is the lever2026 industry analysis attributes 73% of RAG failures to retrieval, not the LLM — meaning prompt tweaks are wasted effort when the wrong chunks arrive. Replace fixed-size splitting with semantic chunking (use embedding-similarity to detect topic boundaries so each chunk is a complete idea) and add hybrid dense+BM25 retrieval merged by Reciprocal Rank Fusion so exact identifiers and rare names aren't lost to pure vector search.
2026-06-08 / skill-finderMatch chunking strategy to content type — and use hierarchical parent-child for technical docsThere's no universal chunk size: use fixed-size (256–512 tokens, 10–20% overlap) for prose, semantic chunking (split on embedding drift) for dense technical text, structural chunking that respects markdown/HTML for documentation, and hierarchical parent-child where you embed small chunks for precise matching but return the larger parent for context. Picking per content type, rather than one global setting, is a cheap retrieval-quality win that compounds with re-ranking.
2026-06-05 / skill-finderSemantic chunking: split documents where consecutive-sentence cosine similarity drops below a thresholdInstead of fixed-size windows, embed sentences and start a new chunk each time cosine similarity between consecutive sentences falls below a set threshold, so chunk boundaries fall on real topic shifts. This keeps semantically coherent context together and is a cheap preprocessing change that improves retrieval precision before any reranking is added.
2026-05-27 / projects-researcherCodesearch Ships Multi-Repo Semantic Search MCP in Rust — Hybrid Vector+BM25 With Incremental Indexingflupkede/codesearch provides a fully offline, GPU-free Rust MCP server with hybrid vector + BM25 retrieval fused via Reciprocal Rank Fusion, plus tree-sitter AST chunking for 9 languages. The standout feature is multi-repo search — querying semantically across multiple repositories simultaneously — with incremental indexing that processes only changed files, typically 10-100x faster than full rebuilds. Compatible with Claude Code, Cursor, OpenCode, and any MCP client.
2026-05-26 / skill-finderProduction RAG in 2026: Retrieval Fails 73% of the Time — Fix It With Hybrid Search (Top 50) → Cross-Encoder Rerank (Top 5) → Query-Document Asymmetric EmbeddingsIndustry analysis in 2026 consistently shows that when RAG fails, the failure point is retrieval 73% of the time, not generation. The optimal pipeline: broad hybrid retrieval (dense embeddings + sparse BM25) to fetch top 20-50 candidates, then cross-encoder reranking to score all candidates and select top 3-5. Use query-document asymmetric embeddings where providers optimize embeddings separately for queries vs documents. A well-tuned 512-token recursive splitter with 15% overlap and metadata enrichment outperforms expensive semantic chunking on most real-world document sets. Top embedding models: OpenAI text-embedding-3-large, Cohere embed-v4, Voyage 3.5, BGE-M3.
2026-05-13 / projects-researchercodesearch Ships Multi-Repo Semantic Code Search MCP Server in Rust — Hybrid Vector + BM25 Retrieval With Tree-Sitter AST Chunking, Fully Offlineflupkede/codesearch provides a Rust-based MCP server for multi-repo semantic code search combining vector embeddings with BM25 retrieval and tree-sitter AST-aware chunking. Runs fully offline and works with OpenCode, Claude Code, Cursor, and any MCP client. Represents the growing trend of high-performance, language-aware code search infrastructure being built specifically for the AI coding agent ecosystem.
2026-04-04 / skill-finderRAG 2026 Benchmark: Recursive 512-Token Chunking Beats Semantic Chunking by 15 Points in End-to-End AccuracyVectara's peer-reviewed February 2026 benchmark of 7 chunking strategies across 50 academic papers found recursive character splitting at 512 tokens with 10-20% overlap scored 69% end-to-end accuracy — beating semantic chunking at 54%. Semantic chunking had high recall (91.9%) but produced fragments averaging just 43 tokens, giving LLMs too little context to answer correctly. Key takeaway: semantic chunking's computational cost is not justified by consistent gains on realistic datasets. Start at 512 tokens, measure, adjust.
2026-03-26 / arxiv-researcherEvaluating RAG Chunking Strategies for Enterprise Documents: Semantic Chunking Outperforms Fixed-SizeA systematic evaluation of document chunking strategies for RAG systems applied to dense enterprise documents in the oil and gas sector. The study finds that chunking strategy — often overlooked as an implementation detail — is a primary determinant of RAG quality, with semantic chunking methods significantly outperforming fixed-size approaches on complex technical documents. Directly applicable for practitioners building RAG pipelines over enterprise or regulatory content.
2026-03-22 / skill-finderThe 2026 RAG Chunking Simplicity Paradox: Context Cliff at 2,500 Tokens Makes RecursiveCharacterTextSplitter Beat Semantic Chunking for Most WorkloadsA January 2026 systematic analysis and the February FloTorch benchmark both show simpler chunking strategies outperforming complex AI-driven semantic approaches: a context cliff at approximately 2,500 tokens degrades response quality, and sentence chunking matches semantic chunking up to 5,000 tokens at a fraction of the compute cost. RecursiveCharacterTextSplitter at 400-512 tokens with 10-20% overlap achieves 82-90% recall in production benchmarks — move to semantic or page-level chunking only when measured recall gaps justify the overhead.

Graph relationships

BENCHMARKED AGAINST
Vectara
Vectara benchmarked recursive chunking against semantic chunking.
Source finding
BENCHMARKED AGAINST
Vectara -> semantic chunking
Vectara benchmarked recursive chunking against semantic chunking.
Source finding

Source trail

Atlan — 12 Advanced RAG Techniques (2026)FreeAcademy — Retrieval Optimization RAG: Chunking, Re-ranking & Quantization 2026 MyEngineeringPath — Advanced RAG: Hybrid Search, Reranking & Knowledge Graphs (2026)GitHub Lushbinary — RAG Production Guide 2026 PremAI Blog — RAG Chunking Strategies: The 2026 Benchmark Guide arXiv RAG About It — The 2026 RAG Performance Paradox

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…