Cluster-based semantic chunking did not beat fixed-size or recursive chunking — a useful negative result

arXiv 2607.01852·medium signal

A July 2, 2026 evaluation (arXiv:2607.01852) tested cluster-based semantic chunking against fixed-size and recursive chunking on long, structured academic theses using the RAGAs framework, and found the sophisticated approach did not outperform the simple ones. Performance varied more with document formatting, preprocessing, and query type (fixed vs. document-specific questions) than with chunking strategy. The authors also flag that RAGAs' faithfulness metric showed limited reliability in this setup — a warning for anyone using it as their primary RAG gate. Practical takeaway: spend your budget on preprocessing and query routing before you buy into semantic chunking.