BRIE: A Clinician-Validated Generator Keeps an EHR Retrieval Benchmark Fresh, and Nine LLMs Still Omit Key Information
arXiv·low signal
arXiv 2609.30205 generates question-answer pairs automatically from longitudinal EHR notes. Nineteen clinicians validated the generator, so the benchmark can refresh itself to avoid leakage and produce multiple acceptable answers. Across nine LLMs and five inference strategies, models often left out clinically important information, especially on questions that required synthesis across several documents and encounters.