You’d think getting wrong advice would make you more cautious. New research says the opposite happens when that advice comes from AI. A study led by Valerio Capraro, a psychology professor at the University of Milan-Bicocca, found that incorrect AI advice made people less accurate, yet noticeably more confident in their answers (via IBM Think).
The study also found that people who received incorrect AI advice became far less willing to admit they didn’t know an answer, even when staying silent was an option, and they were rewarded for accuracy.
Why does wrong AI advice make people so sure of themselves?
Researchers asked participants six questions about obscure movie details, deliberately picking ones the AI model consistently answered incorrectly. This let them isolate what happens when people consult AI, regardless of whether its advice is actually good.
The results were striking. Without AI, participants declined to answer roughly 36 to 44% of the time across two experiments. With AI available, that dropped to just 3 to 6%. Accuracy fell too, from 27.5 percent correct when working alone to just 9.2 percent with AI involved.
Capraro suspects AI may lower the amount of confidence people need before committing to an answer, essentially shifting how they judge their own certainty. Oddly, this held true even when AI advice appeared automatically without being requested, suggesting the effect isn’t just about actively seeking help.
Does raising the stakes change anything?
Adding financial rewards and penalties made people somewhat more cautious and accurate, though the underlying pattern never fully disappeared. Capraro draws a clear distinction here. AI should augment human judgment, not replace it entirely.
He worries this effect extends to children too, since growing up with instant answers could prevent kids from experiencing doubt as part of learning. New reports suggest that the AI-hooked younger generation is sourcing chatbot help even for in-person talks. As Capraro puts it, doubt isn’t a failure of knowledge. It’s often where real knowledge actually begins.