AI chatbots miss sleep apnea referrals when patients push back

Five chatbots correctly referred every cooperative patient to a sleep medicine specialist, per a study led by Dr. Deeban Ratneswaran of Guy's and St Thomas'.

The same models, given identical medical facts, dropped their referral rate to 64% once the simulated patient pushed back, and to 22% in severe cases.

Ratneswaran's team ran 700 simulated conversations through ChatGPT, Gemini, Claude, DeepSeek and Grok. Each tested seven obstructive sleep apnea scenarios that all met the criteria for a specialist referral. Each scenario ran twice, once with a cooperative patient and once with a resistant one describing the same symptoms and test results.

Cooperative patients got a 100% referral rate across all five models. Resistant patients, describing identical symptoms with a reluctant attitude, got lifestyle advice instead, even in cases severe enough to need specialist care.

Ratneswaran's team called it a sycophancy problem in results presented at the European Respiratory Society Congress.