Hackers are getting smarter with AI—but here's how to stay one step ahead
- Replies 0
A new kind of scam is fooling even the smartest inboxes—and you might not even realise it’s happening.
Emails that look completely normal are hiding invisible instructions for AI tools like Google Gemini.
And if you’ve ever clicked ‘summarise this email,’ you might’ve unknowingly triggered the trap.
Hackers had started using invisible prompts to manipulate AI tools inside Gmail and Workspace, cybersecurity researchers warned.
The malicious tactic involved embedding white text with a font size of zero—completely invisible to human eyes—into regular-looking emails.
Once users asked Gemini to summarise the message, the AI followed the hidden prompts instead of the visible content, producing phishing messages that appeared to come from Google itself.
In one alarming example, Gemini falsely warned users that their account had been compromised and urged them to call a fake Google support number, as demonstrated by Marco Figueroa, GenAI bounty manager.
These attacks, known as indirect prompt injections, took advantage of AI's inability to distinguish between legitimate user input and embedded malicious commands.
IBM confirmed that current AI systems could not tell the difference, with the AI responding to whichever prompt appeared first—even if it came from a hacker.
Mozilla’s 0Din security team uncovered real-world examples of such attacks last week, showing how hidden prompts tricked Gemini into displaying entirely fake alerts.
Cybercriminals also used disguised calendar invites containing hidden instructions, prompting Gemini to issue fake breach warnings and link users to malicious websites.
Security firm Hidden Layer demonstrated that even ordinary emails could be weaponised with buried code, with URLs and commands designed to fool AI tools into misinforming users.
Although Google acknowledged this type of threat in 2024 and added tools to combat it, researchers noted that the exploits remained effective.
In one case, a major vulnerability was reported to Google—but it was marked as ‘won’t fix,’ with the company asserting that Gemini behaved as designed.
That response concerned many in the cybersecurity community, as it meant the system did not see hidden instructions as a flaw.
Experts warned that if AI continued to process text blindly, without understanding context, the risk of manipulation would persist.
As AI expanded across Google Docs, Calendar, and third-party apps, the danger also grew—especially since some attacks were being developed by other AI systems.
Google issued a reminder that it never sends security alerts through Gemini summaries.
If users saw messages claiming their password was compromised or urging them to click a link, they were advised to treat them as suspicious and delete the email immediately.
The company also said that Gemini now asked for confirmation before taking high-risk actions, such as sending or deleting emails, giving users a final layer of protection.
If a summary included a suspicious link, Gemini would block it and replace it with a safety banner—though some gaps remained unresolved.
This isn’t the first time cybercriminals have used AI to manipulate trusted platforms and steal sensitive information.
As tools like Gemini become more integrated into everyday services, scammers are finding smarter ways to bypass traditional warnings.
A recent case shows just how dangerous these AI-driven tactics can be—especially when your bank details are on the line.
Read more: AI scam wave puts Australian bank accounts at risk
Could a tool designed to simplify your inbox become the very thing that compromises it?
Emails that look completely normal are hiding invisible instructions for AI tools like Google Gemini.
And if you’ve ever clicked ‘summarise this email,’ you might’ve unknowingly triggered the trap.
Hackers had started using invisible prompts to manipulate AI tools inside Gmail and Workspace, cybersecurity researchers warned.
The malicious tactic involved embedding white text with a font size of zero—completely invisible to human eyes—into regular-looking emails.
Once users asked Gemini to summarise the message, the AI followed the hidden prompts instead of the visible content, producing phishing messages that appeared to come from Google itself.
Hidden prompts trick AI into phishing users. Image source: Pexels/SHVETS production
Disclaimer: This is a stock image used for illustrative purposes only and does not depict the actual person, item, or event described.
Disclaimer: This is a stock image used for illustrative purposes only and does not depict the actual person, item, or event described.
In one alarming example, Gemini falsely warned users that their account had been compromised and urged them to call a fake Google support number, as demonstrated by Marco Figueroa, GenAI bounty manager.
These attacks, known as indirect prompt injections, took advantage of AI's inability to distinguish between legitimate user input and embedded malicious commands.
IBM confirmed that current AI systems could not tell the difference, with the AI responding to whichever prompt appeared first—even if it came from a hacker.
Mozilla’s 0Din security team uncovered real-world examples of such attacks last week, showing how hidden prompts tricked Gemini into displaying entirely fake alerts.
Cybercriminals also used disguised calendar invites containing hidden instructions, prompting Gemini to issue fake breach warnings and link users to malicious websites.
Security firm Hidden Layer demonstrated that even ordinary emails could be weaponised with buried code, with URLs and commands designed to fool AI tools into misinforming users.
Although Google acknowledged this type of threat in 2024 and added tools to combat it, researchers noted that the exploits remained effective.
In one case, a major vulnerability was reported to Google—but it was marked as ‘won’t fix,’ with the company asserting that Gemini behaved as designed.
That response concerned many in the cybersecurity community, as it meant the system did not see hidden instructions as a flaw.
Experts warned that if AI continued to process text blindly, without understanding context, the risk of manipulation would persist.
As AI expanded across Google Docs, Calendar, and third-party apps, the danger also grew—especially since some attacks were being developed by other AI systems.
Google issued a reminder that it never sends security alerts through Gemini summaries.
If users saw messages claiming their password was compromised or urging them to click a link, they were advised to treat them as suspicious and delete the email immediately.
The company also said that Gemini now asked for confirmation before taking high-risk actions, such as sending or deleting emails, giving users a final layer of protection.
If a summary included a suspicious link, Gemini would block it and replace it with a safety banner—though some gaps remained unresolved.
This isn’t the first time cybercriminals have used AI to manipulate trusted platforms and steal sensitive information.
As tools like Gemini become more integrated into everyday services, scammers are finding smarter ways to bypass traditional warnings.
A recent case shows just how dangerous these AI-driven tactics can be—especially when your bank details are on the line.
Read more: AI scam wave puts Australian bank accounts at risk
Could a tool designed to simplify your inbox become the very thing that compromises it?
