OpenAI rolled out Advanced Voice Mode to US and international users several weeks ago, but I was left out of this big ChatGPT (GPT-4o) upgrade because the European Union region wasn’t covered. Fast-forward to last week, and OpenAI quietly rolled out Advanced Voice Mode support to ChatGPT Plus users in the EU, Switzerland, Iceland, Norway, and Liechtenstein.
I haven’t noticed the upgrade in the desktop app, but I saw it late on Friday when I opened ChatGPT on my iPhone 16 Plus to retrieve an older chat about things to do while I was visiting Venice, Italy. Advanced Voice Mode was waiting for me in the ChatGPT app after an app update. I chose a museum to visit on Saturday, thinking I’d test Advanced Voice Mode as a guide that would be there waiting for my questions.
It turns out that Advanced Voice Mode can be a great companion when sightseeing. In fact, I won’t ever step into a museum again without ChatGPT by my side.
Before you get too excited, I also have some bad news. Advanced Voice Mode didn’t work perfectly, and there are a bunch of wrinkles to iron out. But I still think this could be a great way to explore new cities.
This isn’t the first time I’ve used ChatGPT to find out more details about a museum visit. I already knew I’d have questions for the chatbot about the Arsenale and Naval History Museum (Museo Storico Navale) in Venice. But Advanced Voice Mode gave me an entirely new experience.
After configuring Advanced Voice Mode by choosing the voice, I put the iPhone 16 Plus in my pocket and the AirPods in my ear. I already had Background Conversations enabled in the iPhone app, which meant I could talk to the chatbot even after the iPhone display turned off in my pocket.
I started my chat as soon as I entered the museum, and here’s where I encountered the first problem with this scenario. You can’t be too loud in this setting.
I can’t disturb other people around me, but I have to be loud enough for ChatGPT to understand me. Advanced Voice Mode was wrong a few times when trying to pick up my voice. It answered what it thought it heard.
Secondly, Advanced Voice Mode needs a good internet connection to work properly. I realized that as I was waiting for ChatGPT to respond, as I was walking inside a five-story centuries-old building roaming with a VPN on top of a 4G cellular data signal that wasn’t always great.
How did I know? There was some lag in Advanced Voice Mode’s response. At times, ChatGPT took longer to reply. To paraphrase Siri, Advanced Voice Mode “did not quite catch that” more than once. So, the chatbot answered what it thought I asked it to.
I’m telling you all this because this was a spontaneous ChatGPT Advanced Voice Mode experience. I did not consider the potential problems above beforehand. I did not interact with the chatbot before leaving home, thinking it had to “just work.” Looking back, I should have switched off the VPN and chosen a different internet connection to account for connectivity issues.
Also, once inside the museum, I might have held the iPhone out of my pocket to get an idea of the signal strength. Or I could have started asking questions on the higher floors, closer to the windows.
Thirdly, and I’ll always address this ChatGPT issue, AI can hallucinate information. But I don’t care about that risk as much when it comes to history or art. I’m trusting ChatGPT to have ample access to this sort of information. It’s essentially old data probably coming from many different sources. The risk of mistakes should be minimized.
On the plus side, I’ll also say that I switched between languages while talking to Advanced Voice Mode, and ChatGPT complied. Unfortunately, at some point, the chatbot thought I was speaking a different language, and it replied in that language, which I couldn’t understand.
With all that in mind, I loved the idea of talking with what would essentially become a ChatGPT museum guide. I only experienced it for a few minutes until I decided the museum visit would drag too long while I waited for Advanced Voice Mode to understand me or get a better signal.
As I made my way inside Venice’s ample Naval History Museum, I continued using ChatGPT without voice. Instead of talking to it, I took photos of the things I wanted more information about, waited for ChatGPT to answer, and then had the chatbot read them aloud. I never really had to type prompts with photos, as the chatbot would just provide explanations after I uploaded the media.
This isn’t exactly the real-time tour guide that Advanced Voice Mode could offer, but it still uses GPT-4o’s multimodal abilities. The chatbot can understand images, including the text in them.
I am aware that I could always pick museum tours that offer audio guides. But ChatGPT gives me ample room to ask any question I have. It so happens that I continued that conversation over a quick lunch after the visit, asking ChatGPT all sorts of questions about Venice’s history tied to what I had just seen.
I’m certain I’ll load ChatGPT for every museum visit from now on. Overall, this voice experiment was mostly a big failure, yet I’m still thrilled about it. I could interrupt the bot and change the prompt, and ChatGPT would comply. It felt almost like talking to a human, and I’m sure it can get a lot better.
In the meantime, I’ll get better at talking to ChatGPT via voice while I wait for Advanced Voice Mode to improve.
