SmartSearch: Ranking Beats Structure for Conversational Memory Retrieval
arXiv 2603.15599·high signal
Challenges the dominant approach of investing in LLM-based structuring at ingestion time for agent memory systems. Simple ranking-based retrieval at query time matches or outperforms expensive structured pipelines with substantially lower latency and cost. Critical for builders designing agent memory: the costly structuring step may be unnecessary overhead — query-time ranking is sufficient.