As exciting as genAI software might be, it also has side effects that we all need to be aware of. Since AI programs also offer human-like voice modes, it might be easy to have one of these AI models make calls for nefarious purposes.
One such scenario involves an AI impersonating a “very polite and professional” Google representative calling you from a spoofed number. The call is part of a hacker’s attempt to take over your Gmail account. The hack also involves creating fake Gmail recovery emails and fake support emails meant to further convince the victim they’re the target of an ongoing attack.
You might avoid falling prey to the attack if you’re tech-savvy enough. But unsuspecting Gmail users afraid that their account is in danger might end up giving the hacker their password by eventually “verifying” their Gmail account on a fraudulent site.
Sam Mitrovic was one of the targets of a Gmail account takeover hack. Luckily for him, he’s an experienced IT engineer who knew what to look for when prompted with the “evidence” that his account was in danger. He detailed his experience on his blog (via PCMag), explaining the simple steps you should take to reduce the risk of falling for the scam.
Initially, the engineer received a notification to approve a Gmail account recovery attempt that he ignored. Some 40 minutes later, he had a missed call with a “Google Sydney” caller ID.
Exactly a week later, the same thing happened. This was when he decided to pick up the call without realizing he might be talking to an AI made to sound like a human:
It’s an American voice, very polite and professional. The number is Australian.
He introduces himself and says that there is suspicious activity on my account.
He asks if I’m travelling, when I said no, he asks if I logged in from Germany to which I reply no.
He says that someone has had access to my account for a week and that they have downloaded the account data (I then get a flashback of the recovery notification a week before).
Tech-savvy or not, I’m sure this is the step when panic starts creeping in. Mitrovic asked the Google support person to send him an email. The voice said he would:
In the background, I can hear someone typing on the keyboard and throughout the call there is some background noise reminiscent of a call centre.
He tells me that he has sent the email. After a few moments, the email arrives and at a first glance the email looks legit – the sender is from a Google domain.
Thankfully for the IT specialist, he was careful enough to start checking things. While the phone number seemed legit, the email domain looked suspicious. It did not come from a Google server. That’s when he realized he must have been talking to an AI:
The caller said Hello, I ignored it then about 10 seconds later, then said Hello again. At this point I released it as an AI voice as the pronunciation and spacing were too perfect.
I was in the car at this point, parked.
I hung up and drove home to do some more digging.
At that moment it struck me – if it was really an AI call, I could have “reprogrammed” it and prompted it to sing me a song etc.
A callback did not yield any results. Mitrovic investigated the matter further, discovering that other people were subject to a similar scam.
He also made sure nobody accessed his account, as he didn’t find any suspicious activity in his Google account. This proved the claims from the supposed support person were fake.
The point of the whole thing is for the victim to eventually trust the Google rep and agree to verify their account. They would have probably clicked on a link taking them to a Google-like website. But it would have been a scam website meant to grab the password associated with the email account.
The engineer explains the “giveaways” that he was the target of a Gmail account takeover:
- I received account recovery notifications which I didn’t initiate.
- Google doesn’t call Gmail users if you don’t have Google Business Profile connected.
- The email contained a To email address not connected to a Google domain.
- There were no other active sessions on my Google account apart from my own.
- Email headers showed how the email was spoofed.
- Reverse number search showed others who received the same scam call.
If you’re worried about the safety of your Gmail account(s), make sure you set up a strong, unique password for each property. Password managers like 1Password, Apple Passwords, and Proton Pass are your friends. You should also enable Google account passkeys if you can.
Then, when dealing with support calls that might feel like the real thing, remember we’re living in a genAI world where anything is possible. Do not take action in real-time. Instead, ask for them to call you back. Check out Mitrovic’s full blog post on the matter, too, as it contains images that can help you improve your Gmail security practices.
While this attack happened about a month ago, it received more attention recently. It happened around the same time Google launched a new initiative to improve the defense against online scams. Announced on Friday, the new Global Signal Exchange (GSE) initiative is a partnership between Google, the Global Anti-Scam Alliance, and the DNS Research Federation to fight scams and fraud.
Also, it’s unclear whether these Gmail account takeover attempts involving generative AI products inspired Google to take any action. Hopefully, this sort of scam is on Google’s radar for the new GSE initiative.
