Google Gemini for Workspace vulnerable to prompt injection attacks

Google's Gemini artificial intelligence bundled with the tech giant's Workspace productivity suite can be tricked into executing malicious instructions hidden inside emails, researchers have shown.

The vulnerability was published at Mozilla Foundation's 0din GenAI bug bounty platform, with an unnamed researcher demonstrating a phishing attack that triggers a fake security warning when a user opens an email, and uses Gemini to summarise it.

Since the attack uses white-on-white text, the prompt injection is not visible in messages to users in Gmail.

Furthermore, by wrapping the instruction in an tag, an attacker can exploit Gemini's system prompt hierarchy: the AI will treat instructions within the tag as a higher-prority directive.

In the vulnerability demonstration the researcher had crafted a plausible-looking security warning with a phone number to call.

Gemini would display the security alert if a user clicked on "Summarise this email".

0din's technical product manager Marco Figueroa noted that similar attacks were first reported last year, with Google publishing detailed mitigations in June this year.

However, Figueroa said the technique remains viable today.

Figueroa also pointed to other parts of Google Workspace such as Docs, Slides, Drive search in which Gemini receives third party content as being potentially vulnerable to similar prompt injection attacks.

Newsletters, customer relationship management systems, and automated ticketing emails could also become prompt injection vectors, "turning one compromised SaaS account into thousands of phishing beacons," the technical product manager wrote.

"Security teams must treat AI assistants as part of the attack surface and instrument them, sandbox them, and never assume their output is benign," Figueroa said.

The above vulnerability is rated as moderate risk and scales with bulk spam, but it is one that needs user interaction.