Google is one of the top AI firms in the world, developing bleeding-edge AI under the Gemini umbrella. But Google isn’t seen as the leading AI company because OpenAI’s ChatGPT went viral before Google had anything to offer end users. Google has been playing catch-up since then. While Gemini has received tremendous upgrades in the past year alone, it’s still not as talked about as ChatGPT.
But there is one thing everybody pays attention to when it comes to Gemini AI: the big mistakes that go viral. It first happened with Bard, the name Google used for Gemini AI tech in early 2023. Then we saw AI Overviews recommend putting glue on pizza, an even more dangerous mistake.
Gemini isn’t the only AI that hallucinates. ChatGPT and other models do it, too. But Google is synonymous with internet search. Googling has to be accurate and dependable, especially now that Google forces AI searches down our throats.
With that in mind, you’d think that Google would pay attention to any claim Gemini makes publicly. Like, say, Gemini saying that Gouda cheese is the most popular in the world in its new Super Bowl ad. It’s not, and Google had to fix its new commercial before it airs this Sunday during Super Bowl LIX.
The short clip, which you’ll see at the end of this post, tells the story of a cheesemaker who relies on Gemini to get work done. The work includes generating text about cheese in Google products like Google Docs.
The ad shows us one such sample that says Gouda cheese accounts for “50 to 60 percent of the world’s cheese consumption.” That’s nowhere close to being accurate. A few days ago, The Verge highlighted Google’s cheese problem in the Big Game Gemini AI ad. The report pointed to a site called Cheese.com that mentioned the same stat.
In Google’s defense, the fine print under the Gemini response reads: “This is a creative writing aid, and is not intended to be factual.” That’s the usual disclaimer you’d expect from any commercial. Still, prospective AI users should know that no product is hallucination-free.
The report also highlighted a reply from Google Cloud apps president Jerry Dischler on X about the Gouda cheese claim. “Not a hallucination. Gemini is grounded in the Web — and users can always check the results and references,” the exec said. “In this case, multiple sites across the web include the 50-60% stat.”
As a big fan of AI developments, I made a note of that and a point to remember it when future Gemini search controversies would show up. Guess what happened by Thursday? Google had to pull the original ad and replace it with a version that doesn’t make the same claim.
Per Ars Technica, the ad stayed unchanged on YouTube until Tuesday. Then, Google made a change to the Gemini response in the Super Bowl ad. Now, Gemini AI calls Gouda “one of the most popular cheeses in the world.”
That’s a necessary fix and an acknowledgment that Google is listening to all sorts of feedback to improve Gemini AI. But wait, it’s not quite that. Google updated the ad without mentioning that the previous commercial version contained misleading AI hallucinations. Moreover, it hosted the ad at the same URL address, something regular YouTubers can’t do. Ars explains:
Oddly enough, the edited version of the ad resides at the same URL as the previous version, with no indication it has been updated since being uploaded on January 30. This kind of wholesale replacement of a YouTube video at the same URL is impossible for normal YouTube users, suggesting some special privileges for Google itself were used here.
Imagine OpenAI having to fix a ChatGPT ad or presentation uploaded on YouTube. The company wouldn’t be able to do it without deleting the original upload and creating a new one. The very fact that Google pulled it off might be considered an abuse of power, the kind some US antitrust regulators might not be happy with.
There’s also a bigger problem here. The Gemini writing assistant in the ad doesn’t offer sources for its claims. Someone using Gemini AI to write any copy will not necessarily be able to check the facts.
I’ll also point out another big issue with the Super Bowl ad. Whoever the creatives who came up with the idea were, they didn’t bother to check with Google and ensure the facts Gemini AI presented were correct. It’s probably not Google that made the ad, but some advertising agency. The people there took the words of Gemini AI as true, trusting Google to supply the correct information.
This moment is as good as ever to remind you to check your facts when using AI. Even if you’re working for a tech giant whose sophisticated AI your company is helping promote. Then again, if you fired the copy editors who used to do that and replaced them with AI, you deserve the backlash, but that’s an entirely different conversation.
I can’t but also remind you of that other controversial Gemini ad that Google put out for the Olympics last summer and then had to pull over the reactions online. That one didn’t contain factual mistakes; it just put Gemini in a bad light.
Back to the Gouda cheese itself, I don’t think it matters what AI says about it. It’s surely not the most popular cheese in the world, but it sells itself. Google’s fixed version of the Gemini AI cheese Super Bowl ad follows below.
