Move over Google Assistant; Google has a brand new personal AI assistant that it’s much cooler and more useful than anything it did before. It’s called Project Astra, and it’s powered by Google’s Gemini AI.
Project Astra is an agent-type AI product that uses Gemini’s multimodality to answer real-time questions about the world around you.
It sounds a lot like OpenAI’s new GPT-4o upgrade that ChatGPT just got. That’s because it is. But Project Astra has at least one exciting advantage over ChatGPT that was obvious from the first demo we saw at Google I/O 2024.
Unlike OpenAI, Google did not give us a live demo of Project Astra. But we got something almost as good: A recorded demo of a live experience with Project Astra running on the phone.
Just like GPT-4o, Project Astra can see the world around you and answer questions about it. In Google’s demos, the AI looked at code but also identified objects in a room that could play music.
That opens up a whole new set of use cases for genAI assistants. You’ll be able to point your smartphone and ask questions about the world around you. This is even better and easier than Circle to Search, an AI feature that Google unveiled earlier this year.
The real mind-blowing detail in Google’s Project Astra demo concerns a feature GPT-4o doesn’t have: keeping track of your surroundings. If you need to ask questions, the AI remembers what it saw in the video.
At one point, the smartphone user asks the AI where their glasses are. And Project Astra gives them the correct answer.
It’s all happening very fast, which is another strength of this Gemini feature. If Google can achieve this speed, Project Astra agents will be incredibly useful. They’ll provide the information you want right when you want it.
I’ll also point out that Project Astra is the kind of AI functionality I’d want to use on smart glasses, the kind I’m dreaming about replacing my iPhone. It would be even easier to use AI on AR glasses, but I’m speculating at this point. Google also demoed such a feature briefly at the end of the clip above.
Some of these Project Astra agents will be available via the Gemini app later this year, but you won’t be able to use them right now. I can’t help but wonder whether Project Astra agents will initially be exclusive to the Pixel 9 phones coming out this fall.
