AI advice made people three times less accurate but twice as confident, researchers found - The Next Web

What the report says

The Next Web reported that a study by researchers at three French and Italian universities found that access to AI advice made participants less accurate while increasing their confidence. According to the article, people’s willingness to answer “I don’t know” fell from 44% without AI to 3% with AI. Accuracy declined from 27% to 9%, while confidence rose from 30% to 76%.

The study was authored by Valerio Capraro of the University of Milano-Bicocca, Chiara Marcoccia of École Normale Supérieure, and Walter Quattrociocchi of Sapienza University of Rome. The researchers used questions on visual details from films, a category where the AI model used in the experiment, Step 3.5 Flash, was typically wrong. The Next Web said this design was intended to test whether people would defer to unreliable AI rather than simply make reasonable use of a dependable tool.

The article reported that some participants who would have answered correctly on their own changed to incorrect answers after consulting the AI. Monetary incentives produced only limited improvement: willingness to admit uncertainty increased to 8% and accuracy to 16%, still below the no-AI results cited by the publisher.

The findings matter because they point to a risk that AI systems can discourage users from recognizing uncertainty. The Next Web connected the study to wider concerns about “cognitive surrender,” a term it said Wharton researchers used for people accepting incorrect AI answers while feeling more confident. The article also noted worries about children and education, where AI-generated answers may be presented with confidence even when they are wrong.