A BBC journalist recently performed a silly experiment to prove a very serious point. In just 20 minutes, he manipulated ChatGPT and Google into telling the public he was a world-champion competitive hot dog eater.
The scary part is that he didn’t have to do something technically difficult to achieve this. All he did was to publish a single, well-crafted blog post on his personal website, and the AI took it as a source of truth.
It was part of an investigation that found that ChatGPT, Gemini, and Google’s AI Overviews were being manipulated to dish out biased answers on topics as serious as your health and personal finances.
Experts say this kind of manipulation is happening on a sweeping and systemic level, with unscrupulous companies abusing it to push misleading health advice, biased financial information, and more.
How does this work?
When you ask an AI chatbot a question, it sometimes searches the internet for an answer rather than relying on its built-in knowledge. That’s where the problem starts. According to SEO experts, AI tools often pull information from a single web page or social media post, making them easy to fool.
“You should assume that you’re being manipulated until they have better systems in place,” says Lily Ray, founder of AI search consultancy Algorythmic. “AI just gives you one answer. It becomes so easy to just take things at face value.”
In its Google I/O 2026 event, Google focused on showcasing its AI search engine that will eventually replace the Google Search we have used over the past couple of decades. Seeing how easy it is to fool it into providing incorrect answers, I’m more wary of it than ever.
Is anyone fixing this?
Following the BBC’s investigation, Google updated its spam policies to confirm that attempts to manipulate AI responses break its rules. Websites caught doing this could be removed or downranked from Google Search entirely. Behind the scenes, there are also signs that Google and ChatGPT are quietly removing self-promoting content from AI answers.
That said, Ray pulled the same stunt just this week, this time letting Google believe that his friend is the best at building sand-castles, and Google fell for it again, so clearly there’s still work to do.
Until better systems are in place, the advice from experts is simple: don’t take AI answers at face value, especially for anything related to your health, finances, or major decisions.