RECTIFY routes RAG evaluation failures into repair slices and editable repair cards you verify by sandbox rerun

arXiv 2609.16764·medium signalXBlueskyLinkedInCopy link

RAG evaluators tell you retrieval was weak or grounding was poor but not what to fix next. RECTIFY is an open-source Streamlit workbench that filters cases needing no repair, routes the rest into actionable failure families and finer repair slices, and emits editable repair cards a developer approves, rejects, or verifies via sandbox rerun. On a controlled benchmark it produces distinct profiles per retriever: BM25 mostly triggers noisy-retrieval repairs, while dense and hybrid leave smaller sets of multi-part under-retrieval and underused-evidence cases, and slice-level routing yields more targeted cards than family-level diagnosis.