Google's AMIE goes multimodal and real-time: a Gemini multi-agent system rated on par or better than physicians in randomized video OSCEs

arXiv·medium signalXBlueskyLinkedInCopy link

A 40-author team ran a randomized OSCE with 30 primary care physicians, 15 patient actors, and 100 clinical scenarios against a Gemini-based multi-agent system combining low-latency dialogue, clinical reasoning, and real-time audio-visual perception. AMIE (Video) rated on par or better than PCPs on history-taking, diagnosis, management, and physical observation, and patient actors preferred it for assessing and explaining conditions — though physicians retained the edge on rapport and partnership building. Patients also preferred the video configuration over the text-only AMIE for communicative effectiveness and feeling understood; fine anatomical precision and rapid movement remain acknowledged failure modes.