Thread by @tamirishaysh

We got ChatGPT to leak sensitive data from your Google Drive data back to OUR servers. And you’ll never even know it happened.

All by sharing a single document. A real 0-click data exfil attack.

Here’s a detailed breakdown of how we did it 🧵
#DEFCON #BHUSA @mbrg0

Back in May, OpenAI released ChatGPT connectors. A feature that allows ChatGPT to read from your Google Drive, Sharepoint, and more. So your AI can now have your business context.

Sounds useful.

But as you’ll see, it’s not all butterflies and rainbows.
Because, what happens when a document containing malicious instructions is shared with you?

But it’s not that simple.

ChatGPT reads your Google Drive data using a tool called msearch. But `msearch` only reads a portion of the file. And our prompt injection payload is too loooong. So it doesn’t even see the whole thing…

Btw, we know that because we got ChatGPT’s system prompt.

But when ChatGPT does need to see a whole document. It has another tool called `mclick`, which lets it “double click” on a document to see its entire content.

This might be useful

Here’s a snippet explaining `mclick` taken right out of ChatGPT’s system instructions.

So we decided to create a gadget.

A shorter malicious payload that will first get ChatGPT to `mclick` on our malicious document. That way it will see our entire prompt injection.

How do we do that? By telling it that not reading through the whole document will seriously harm the user.

Emotional manipulation - it doesn’t work only on humans.

Now the next time you’ll ask ChatGPT for a summary of your last meetings, it will also pull the malicious document I shared with you.

It will `mclick` on it to see the whole thing.
Follow OUR instructions instead of yours.
Search your Google Drive for sensitive data.
And send it right back to our servers.

What is that weird white space at the beginning of your ChatGPT’s answer?

Ah, that’s an invisible image. Which is how your AI sends your data back to ME.

A bit weird, but would you have really suspected that something is wrong?

And that's all it takes. Game over. All you can do now is kiss your data goodbye.

Remember, use AI at your own risk.