Study: AI Advice Cut Accuracy From 27% to 9% While Raising Confidence From 30% to 76%

Hacker News / TNW·high signal

Researchers at University of Milano-Bicocca, École Normale Supérieure and Sapienza found that participants given AI assistance answered correctly 9% of the time versus 27% in the baseline group, while self-reported confidence rose from 30% to 76%. The sharpest finding is on intellectual humility: willingness to answer 'I don't know' collapsed from 44% to 3%. The authors frame this as 'cognitive surrender' — adopting model output with minimal scrutiny, overriding both intuition and deliberation — and the HN thread (349 points, 202 comments) largely turned into practitioners arguing about whether agent harnesses systematically train this failure mode into their operators.