Google unveiled four Pixel 9 phones on Tuesday, including three Pro models, of which one is foldable. This is the boldest line of Pixel devices Google has ever released and it’s no coincidence it’s happening as generative AI booms. The Pixel 9 phones don’t just prove that Google can make phones that are at least as compelling as the latest iPhones. The Pixel 9, 9 Pro, 9 Pro XL, and 9 Pro Fold are all vessels that will carry Google’s vision of AI.
Google’s Gemini models power this vision, from the smaller Gemini Nano, which runs on-device artificial intelligence features, to Gemini Advanced, which powers sophisticated features.
To mark the occasion, Google announced various new Gemini AI features for the Pixel phones on Tuesday, including Gemini Live. This is the multimodal, sophisticated voice mode chatbot Google demoed at I/O in May. It’s similar to ChatGPT’s Voice Mode, which actually stole Google’s thunder by launching a day before the I/O keynote.
Gemini Live
Gemini Live requires Gemini Advanced to work. That means you will need a $20/month subscription to Google One to access the new feature on your devices. Pixel 9 Pro orders will come with a free year of Google Advanced access, including Gemini Live.
With Gemini Live, you talk to the AI just like you’d talk to a human. The AI can keep track of the discussion even if you interrupt it. Gemini Live will support multimodal input, including text, voice, and imagery. You’ll have to wait for camera support however, as that’s only coming later.
Right now, you’ll have ten voices to choose from, though Gemini Live will only work in English at launch.
Gemini Live will be available in the Gemini app on Android devices, including Pixel 9 phones. Later, it will be available on iPhone via the Google app.
Gemini Live will work in the background even when the Pixel’s screen is off. During the Pixel Buds Pro 2 segment, Google showed that you’ll be able to use the new earphones to talk to Gemini Live.
Gemini Assistant
Gemini will power the Google Assistant experience on Pixel 9 phones. Just say, “Hey, Google,” or press and hold the power button to invoke it. This is a big upgrade for the Assistant, which will use Gemini AI to answer whatever you throw at it, including questions related to what’s currently on the screen and data in your other Google apps
This is not to be confused with Gemini Live. Gemini Assistant will work on all devices where it’s available without requiring a subscription to Gemini Advanced.
Google says it plans to enhance the capabilities of Gemini Assistant with new models like the Gemini 1.5 Flash update. This should improve the speed and reliability of responses.
Gemini contextual overlay
One of the great features coming to the Assistant is the ability to look at your screen and provide contextual information. For example, you might have questions while watching a YouTube video about topics you like. You could invoke Gemini AI to ask those questions.
Google announced new AI features for the Pixel 9 phones, including the Gemini Live voice mode that brings natural conversations to the AI assistant.
The feature will be available in the Gemini and Gemini Advanced mobile apps in the coming weeks. It isn’t likely to come to iPhones, considering that it requires access to the display.
Gemini in more apps and services
Google announced new Gemini extensions during the Pixel 9 event. These extensions will let Android users ask Gemini about information in other Google apps. The list includes Keep, Tasks, Utilities, Calendar, and YouTube Music. For example, you’ll be able to ask Gemini for information about your calendar, while also showing the AI a list of concert dates for your area.
Gemini will support even more complex prompts that combine data from multiple Google apps. Here’s an example from Google:
Let’s say you’re hosting a dinner party: Have Gemini dig out that lasagna recipe Jenny sent you in your Gmail, and ask it to add the ingredients to your shopping list in Keep. And since your guests are your college friends, ask Gemini to “make a playlist of songs that remind me of the late ’90s.” Without needing too many details, Gemini gets the gist of what you want and delivers.
In the coming weeks, the extensions will be available in Gemini, Gemini Advanced for the web, and Gemini Advanced mobile.
Pixel Screenshots app
All Pixel 9 phones will ship with a preloaded Pixel Screenshots app. Gemini will gather data from it if you decide to use it, with everything processing on-device via the Gemini Nano model.
Gemini will then be able to retrieve information from your screenshots when you ask questions about them. This is a handy way to handle the increasing amount of data we collect via screenshots and a feature I hope Apple Intelligence will also offer on the iPhone.
Add Me photo mode
Also launching on Pixel 9 in preview is a new generative AI feature called Add Me. I’m not a big fan of creating photos of events that never happened, and Add Me falls in that category. But I can see how some Pixel 9 users will love using it.
With Add Me, you can ensure everyone is in the picture, including the person taking the photo. You’ll need to take two photographs for that. First, person A takes the group photo. Then, person B from the group comes to take a second photo, while person A goes to the group and takes a designated position. The Pixel phones will merge the two images.
Auto Frame
This is another camera mode that lets you use AI to crop photos automatically. Auto Frame will be available in Magic Editor.
Reimagine
Speaking of AI photos, a Reimagine AI feature in Magic Editor will let you change the background of your pictures on the fly. Want pink clouds in the background? You got it.
Pixel Studio
Pixel Studio is another fun generative AI feature for Pixel 9 owners to enjoy. As the name suggests, you’ll be able to generate images on your device via text prompts. Input your instructions and then select the desired output. You’ll also be able to edit your pictures in various styles.
What about privacy?
I care most about privacy when it comes to advanced AI functionality like the ones Google is promoting with these new Gemini features. Google also detailed the privacy protections built into Gemini during its Made by Google event.
Google does not share any data you input into Gemini with a third-party AI provider. However, Google will send data to its servers where some of the Gemini processing occurs. Interactions from Gemini Live chats, which require Gemini Advanced processing, are one example.
On-device AI processing on Pixel 9 phones means no data will reach Google’s servers. The new Pixel Screenshots app is a new AI feature that runs locally via Gemini Nano.
