I give it 3 months until Sora 2 is used to generate a video of a well known executive saying horrible things to tank a company’s stock.
We’re going to see impact in the stock market from believable AI video that we’ve never witnessed at scale.
Prep your team, family and friends.

https://twitter.com/yuchenj_uw/status/1973076229193396542

If everyday folks don’t understand that this level of believable AI generated video and audio content is currently possible, then they could fall for it.
If they know it’s possible and know to verify authenticity, we have a possibility to keep folks safe.

Now can you use the ChatGPT Agent to:
- download malware instead of that free software you were looking for online
- accidentally leak your emails to the public
- inadvertently share your private photos to social media
- book a nonrefundable $10k first class flight to Europe

https://twitter.com/sama/status/1945900345378697650

In addition, your ChatGPT Agent can also:
- Reply weirdly to your family, colleagues, & friends in messages, confusing them deeply
- Misunderstand an important opportunity that comes in via email and turn it down
- Negatively impact M&A with strange emails found in discovery

My favorite way to hack in my ethical hacking is phone call based hacking with impersonation. Why? Because it has the highest success rate. This is what we're seeing in the wild right now, too.
Let's talk about how phone call attackers think and how to catch Scattered Spider style attacks for Insurance companies (that are heavily targeted right now, Aflac recently)
1. *Impersonating IT and Helpdesk for passwords and codes*
They pretend to be IT and HelpDesk over phone calls and text message to ask for passwords and MFA codes or credential harvest via a link
2. *Remote Access Tools as Helpdesk*
They convince teammates to run business remote access tools while pretending to be IT/HelpDesk
3. *MFA Fatigue*
They will send many repeated MFA prompt notifications until the employee presses Accept
4. *SIM Swap*
They call telco pretending to be your employee to take over their phone number and intercept codes for 2 factor authentication Let's talk about the types of websites they register and how to train your team about them and block access to them.
Scattered Spider usually attempts to impersonate your HelpDesk or IT so they're going to use a believable looking website to trick folks.
Often times they register domains like this:
victimcompanyname-sso[.]com
victimcompanyname-servicedesk[.]com
victimcompanyname-okta[.]com
Train your team to spot those specific attacker controlled look-alike domains and block them on your network.

If a user’s expectations about how a tool functions don’t match reality, you’ve got yourself a huge user experience and security problem.
Humans have built a schema around AI chat bots and do not expect their AI chat bot prompts to show up in a social media style Discover feed — it’s not how other tools function.
Because of this, users are inadvertently posting sensitive info to a public feed with their identity linked, including prompts with:
- exact medical issues
- federal crimes committed
- tax evasion
- home address
- interest in extramarital affairs
- sensitive court details
- private photos of unclothed children
- audio asking personal questions
- private upcoming travel plans
- questions about the legality of actions
- challenges in personal relationships
- feeling shame with disabilities What do I recommend as next steps for Meta and other orgs considering a public AI chat bot prompt feed?
1. Pause the public Discover feed. Your users clearly don’t understand that their AI chat bot prompts have been made public.
2. Ensure all AI chat bot prompts are private by default. This goes for all future AI chat bots as well. Don’t wait for users to accidentally post their secrets publicly. Notice that humans interact with AI chatbots with an expectation of privacy, and meet them where they are at.
3. Alert users who have posted their prompts publicly and that their prompts have been removed for them from the feed to protect their privacy.

Yes and also, interestingly, this tool takes all of the most commons steps used to hack people & companies — from OSINT (open source intelligence) via social media, to target selection, to pretext development, to contact + phishing — and automates it completely for attackers.

https://twitter.com/savannahfeder/status/1890487425425711566

Imagine an attacker uses this tool to seek out people discussing their anger about working at a specific company they are employed by (rather than for its intended purpose of finding competitor’s users who are complaining), seeks those individuals out in an automated fashion, and delivers a believable and potent phishing lure via automated message.
Sure, this tool can be used for marketing but it can also be used for highly targeted phishing campaigns, and phishing is how many hacks begin.

I just live hacked @ArleneDickinson (Dragons' Den star - Canada's Shark Tank) by using her breached passwords, social media posts, an AI voice clone, & *just 1 picture* for a deepfake live video call.
Thank you @ElevateTechCA @Mastercard for asking me to demo these attacks live! What are the takeaways from this Live Hack with Arlene?
1. Stop reusing passwords - when you reuse your password and it shows up in a data breach, I can then use that password against you everywhere it's reused online and simply log in as you stealing money, access, data, etc.

LinkedIn is now using everyone's content to train their AI tool -- they just auto opted everyone in.
I recommend opting out now (AND that orgs put an end to auto opt-in, it's not cool)
Opt out steps: Settings and Privacy > Data Privacy > Data for Generative AI Improvement (OFF) We shouldn't have to take a bunch of steps to undo a choice that a company made for all of us.
Orgs think they can get away with auto opt in because "everyone does it".
If we come together and demand that orgs allow us to CHOOSE to opt in, things will hopefully change one day.

We now live in a world where AI deepfakes trick people daily.
Tonight we see @TaylorSwift calling out the use of an AI deepfake of her falsely endorsing a presidential candidate.
Let’s talk thru how to spot deepfakes in videos, audio (AI voice clones / calls), & social media… First, let’s start with an experiment!
Do you think you can reliably spot fake AI images? Here’s an opportunity to see how well you can spot AI generated photos (some are easy and some are more challenging):

Next we’ll talk about tips to spot deepfakes.detectfakes.kellogg.northwestern.edu

Let’s talk about risks w/ Apple’s new camera button & Visual Intelligence AI tools + integrations -- the potential ability to learn a stranger’s identity by simply taking a picture.
Without big 3rd party integration guardrails, this new camera button + AI could invade privacy. Yes, it’s exciting to be able to snap a pic and learn what you see with AI!
Within the Apple ecosystem, there are guardrails to prevent those AI tools from invading privacy. For example, if you upload a pic of a person to the integrated ChatGPT it refuses to tell you who it is.

We are currently in one of the largest global IT outages in history.
Remember: verify people are who they say they are before taking sensitive actions.
Criminals will attempt to use this IT outage to pretend to be IT to you or you to IT to steal access, passwords, codes, etc. How will this hit everyday folks at home?
Please tell your family and friends:
If you receive a call from “Microsoft Support” about a blue screen on your computer, do not give that person your passwords, money, etc. A criminal may ask for payment to “fix” the blue screen for you.

This AT&T breach will massively disrupt everyday folks, celebrities, politicians, activists…
The breach includes numbers called/texted & amount of call/text interactions, call length, & some people had cell site id numbers leaked (which leaks the approximate location of user).

https://twitter.com/josephfcox/status/1811709707674955928

1. What can a criminal do with the data stolen: Social Engineering

The believability of social engineering attacks will increase for those affected bc attackers know which phone numbers to spoof to you. Attackers can pretend to be a boss, friend, cousin, nephew etc and say they need money, password, access, or data with a higher degree of confidence that their impersonation will be believable.

Spoofing (changing caller ID) takes less than a minute and can be done using apps available on the App Store.
Here we see Mark Cuban talking about getting tricked through a phone scam where the attacker spoofed a Google number (Google assistant) and took over his Gmail account🧵 The scam is simple, here’s the breakdown for your family, friends, team, etc with an example video at the bottom:
1. Attacker finds your phone number in data breach or on data brokerage site
2. Attacker sets up which phone number to display on your caller ID with a spoofing app from the App Store (cheap and simple)
3. Attacker places call to victim and pretends they’re with Customer Support (in this case, recovery support at Google), which displays a “Google” number on victim’s caller ID
4. Attacker says there has been an incident on your account and to follow the steps with them to recover access
5. Victim gives attacker details like password, MFA code, or account recovery details to “protect the account from compromise” (in reality, this is the attack itself, of course)
6. Attacker takes over the account and now can do anything victim used to be able to do on account (email in threads and attack others, request fraudulent wire transfers, steal all data, etc)
7. Typically the victim struggles to regain access to their account and the attacker hits many on their contact list
8. Because Mark Cuban is who he is, he was able to regain access with special support that most others would not receive.

Example spoofing phone call attack video below:

Next, let’s discuss how to prevent yourself or others falling for this attack.

https://x.com/racheltobac/status/1627843252085993473?s=46&t=NyA6Hhhifs99eqUAKlHl2Q

I need to explain how AI text to sound effect can be used by criminals to make their kidnapping or bail-related scams believable.
In the wild, we’re seeing “female scream” or “young boy screaming specific name” used in AI voice cloning phone attacks with phone number spoofing 🧵

https://twitter.com/elevenlabsio/status/1796567542565118151

We’re already seeing these generic scream-related sound effects used in criminals pretexts to convince the call receiver that their loved one is truly in trouble and to send money without question.
As AI evolves, we’ll see more and more believable and specific sounds in use.

lol leveraging this real time translator to phish via phone calls in the target’s preferred language in 3…2…

https://twitter.com/tomwarren/status/1790074556981403997

So far, AI has been used for believable translations in phishing emails — E.g. my Icelandic customers are seeing a massive increase in phishing in their language in 2024. Before only 350,000 or so people comfortably spoke Icelandic correctly, now AI can do it for the attacker,

How was social engineering potentially used in the open source security vulnerability? And more
Let's break down the open source security vulnerability (in xz/liblzma) that caused massive upstream supply chain risk & could have allowed the attacker the ability to compromise machines, steal data, attack other machines, destroy info, etc. 1. Was the contributor Malicious, Compromised, or Coerced? Let’s talk Occam’s Razor
We don’t know everything yet so let’s take into consideration Occam’s Razor -- the simplest explanation is the most likely explanation so it’s likely that the contributor of the vulnerable code is the malicious actor themselves, but we can acknowledge it’s possible that he was coerced or compromised.
However, it’s important to note this contributor tried to get this library included in the Fedora Linux Distro and worked with them diligently to fix it to their liking, so it seems unlikely to many that it was a simple account compromise/account takeover.

Everyone's talking about the woman who put $50k in a shoebox and handed it to scammers and how they wouldn't fall for it. It would shock you how many everyday people have embarrassing scam stories.
I can't help once the scams over, but I can help you Spot The Scam upfront: Let’s go thru the anatomy of a scam, we’ll call this The Shoebox Scam.

1. Spoofing customer service:
we’ve seen a major increase in attacks starting with a phone call in the last few years. It’s easy to make the caller ID say anything with spoofing technology you can download on the App Store. Spoofing takes less than 30 seconds to set up and costs about a dollar per call.
When hacking, we often pretend to be someone trying to help you, to encourage you to give up sensitive info about yourself, your account, your money, etc.

Notice that the attacker was a polite woman to start off this scam, not mean and cruel but helpful. Mimicking real customer support interactions when loss prevention calls.
thecut.com/article/amazon…

AI text-to-video is here and we need to discuss the risks.
They mention in this thread that they’re considering the ways adversaries would leverage this content to harm thru red teaming but I’m still concerned.
My biggest concern is how this content could be used to trick, manipulate, phish, and confuse the general public.
- for example, imagine an adversary uses this tool to build an AI video that appears to show a vaccine side effect that doesn’t exist
- imagine an adversary uses this tool to show unimaginably long lines in bad weather to convince people it’s not worth it to head out to vote that day
This tool is going to be massively challenging to test and control under many, let alone most, adversarial conditions. In their post, they discuss rules they’re implementing to limit adversarial use of this text-to-video tool like limiting extreme violence, celebrity likeness, hateful imagery, etc.
But take my example above, prompting this AI tool for “a video of a very long line of people waiting in a torrential downpour outside a building” isn’t in violation of these policies — the danger is in how it’s used.

If that AI generated video of an impossibly long line of people in torrential downpour is used by an adversary to post on social media on the Election Day, now it could be used to convince certain folks to stay home and avoid the polls and line/weather.

*Account Takeover Prevention Guide*
If you watched the SEC account hack that moved markets yesterday & wondered how to prevent account takeover for your personal, business, or high profile social media account, here's an Account Takeover Prevention Guide for you and/or your org. 1. *Remove phone number from Twitter account to prevent SIM swap and account takeover through phone number password reset flow*
Settings and Support > Settings and Privacy > Your Account > Account Information > Remove Phone Number in Phone Number field
On Twitter specifically, there's a vulnerability affecting higher profile accounts or accounts with the new style of checkmarks. If they want to be "verified" they have to add a phone number to do so, but this significantly increases risk for their account due to SIM Swapping.
Many don't realize that when they add the phone number to become "verified" it *stays* on their account. Once that phone number is on the account, they then don't realize that the phone number can be used to completely reset the account's access through the phone number password reset flow (seen below in photo) and here online:
Criminals can takeover your phone number through a process calling SIM Swapping (calling your telephone company and taking over your phone number and account by pretending to be you).
This then allows the criminal to reset your password through the attacker controlled phone number, or siphon out SMS 2-factor codes sent to that phone number.
This is what Twitter effectively claims happened to SEC yesterday when their phone number was taken over to reset an account "without any 2 factor authentication" on the SEC account.help.twitter.com/en/managing-yo…

Thank you @lorenzofb @TechCrunch for discussing the 23andMe intrusion with me.
Here’s a breakdown thread with more of my thoughts:
1. What happened in this data leak?
2. What can orgs proactively do to prevent similar intrusions?
3. What can individuals do to limit their risk of account takeover in similar ways on other sites?

https://twitter.com/lorenzofb/status/1711746596826653182

1. What happened in this data leak?

Cyber criminals were able to find passwords that were involved in other breaches online and use a method called “credential stuffing” to attempt those previously breached and reused passwords on 23andMe to login as other users.
Unfortunately, most folks reuse their passwords across many sites and apps and when those passwords are stolen they can be used to gain access to your account anywhere else the password is used online.
The attackers took the passwords from other breaches, stuffed them into 23andMe and then used an opt-in feature called DNA Relatives to enumerate genetic data of similar groups.
23andMe doesn’t yet appear to be hacked itself, rather the formerly breached passwords reused by the 23andMe users allowed the attacker to gain access to user accounts by logging in as the user and stealing sensitive genetic data.

One of the easiest ways for me to hack is simply:
1. Look up who works at a org on LinkedIn
2. Call Help Desk (spoof phone number of person I’m impersonating)
3. Tell Help Desk I lost access to work account & help me get back in

I hope we learn more & get confirmation of methods

https://twitter.com/vxunderground/status/1701758864390050145

The threat actors claim this was their attack method to compromise MGM Resorts. I’m sure we’ll learn details soon.
For now I’ll say that the attack method they claim worked for them does indeed work for me often. Most orgs aren’t ready for phone based social engineering.

Here’s how I used AI to clone a 60 Minutes correspondent’s voice to trick a colleague into handing over her passport number. I cloned Sharyn’s voice then manipulated the caller ID to show Sharyn’s name with a spoofing tool.
The hack took 5 minutes total for me to steal the info.

https://twitter.com/60minutes/status/1660428419438354435

How to stay safe?
1. Make sure your folks know that caller ID is easily faked. Voices can also now be impersonated.
2. If they receive a dire call from “you”, verify it’s really you w/ another method of communication (text, DM, FT, call, etc) before action (like sending money).