One of the biggest immediate worries with AI products like ChatGPT is not that they’ll take over the world, eradicating humanity in the process. It’s that they’ll distort the truth through hallucinations or answers to prompts that are factually incorrect or misleading.
That’s why hearing that Google just faked the amazing Gemini demo we saw a few days ago is incredibly disappointing. No matter how Google tries to explain it or how accurate Gemini might be at responding to the prompts shown in the clip in real life, it’s still a massive fake. The company showed off a product that doesn’t exist. Or a product with capabilities that have been greatly exaggerated.
It turns out that Google might be so incredibly desperate when it comes to matching ChatGPT that it needs to mislead us. And, again, that isn’t very reassuring from a company I want to trust with the safe development of AI.
First, let’s take a look at the following video, titled “Hands-on with Gemini: Interacting with multimodal AI.” The clip had over 1.66 million views and over 44,000 likes at the time of this writing. The number of dislikes isn’t shown, as usual for a YouTube clip these days.
The clip’s description does indicate that what we’re about to see isn’t fully real. “For the purposes of this demo, latency has been reduced and Gemini outputs have been shortened for brevity,” Google says. On the one hand, that’s understandable. On the other hand, this is the first red flag. The whole video makes it look like Gemini is incredibly fast.
Google does say this at the beginning of the clip: “We’ve been capturing footage to test it on a wide range of challenges, showing it a series of images, and asking it to reason about what it sees.” But it’s still not good enough. The video is misleading.
Google doesn’t say that Gemini has no support for voice input. And what’s amazing about the clip is that Google made it look like the user was talking with Gemini in real-time. That wasn’t the only misleading thing. Google also made it seem like Gemini was getting video input in real-time, being able to react to what was happening on that table.
The reality of Gemini is nowhere near as impressive. Google used text prompts for the clip. And it showed the AI images. I will say again that the AI still delivered results, which proves it’s not all snake oil. Gemini works, but it doesn’t work like it does in the video. Not even close
Bloomberg was the first to reveal the video to be a fake. Google admitted to Bloomberg that the video didn’t happen in real-time. However, Google has not updated the YouTube clip to make the same disclosures to people watching the demo online.
Google apparently disagrees. It pointed The Verge to a post from Oriol Vinyals on Twitter/X. The vice president of research and deep learning lead at Google’s DeepMind explains how they made the video, and that it isn’t real. He says Google made the video to inspire developers.
However, the same disclosure does not appear in the video itself. And that’s misleading and very sad for a company like Google.
The product in the video would be a massive upgrade over ChatGPT. Being able to talk like that to the AI while it’s looking at a real-time video feed is amazing. It’s the kind of tech that could power devices like the Humane Ai Pin or Apple’s Vision Pro. I’m sure we’ll have that in the not-too-distant future. And Google will be among the first to have it on its devices.
But the way Google went about demoing Gemini is disingenuous. It makes me question all the Gemini clips that Google released. Everything Gemini-related is vaporware until proven to be accurate in the real world.
Also, yes, every tech company crafts careful stories in demos and marketing clips. But it’s one thing to fake press photos for a phone and quite another to fake the abilities of an AI product, considering all the worries about AI misuse out there.
Why imply Gemini is so amazing when it’s clear that anyone who likes testing generative AI software will try to replicate that video? They’ll soon learn that Bard, SGE, the Pixel 8, and any other Google AI product based on Gemini can’t deal with voice like that. Nor can it watch real-time events unfold.
I did say a few days ago that Google going for a virtual Gemini launch would make it impossible to test Gemini by the real people in attendance. It looks like that was correct. Google might have done this solely for the sake of keeping investors happy. Gemini might not truly be ready for a commercial debut. That just means I’ll stick with ChatGPT for the time being. Maybe forever.
