Thread by @invtym

#usenix and now: Creating, Weaponizing, and Detecting #DeepFakes

#usenix #DeepFakes Image manipulation is pretty old. Mussolini had a horse handler edited out of an image, Stalin had people edited out of photos, and so on

#usenix #DeepFakes Photo editing is fun! We can faceswap Barack and Michelle Obama! And for a while skilled people could make a fake image, but it was difficult for unskilled people to do.

#usenix #DeepFakes Now we can easily generate people who do not exist - and you don't even have to know Photoshop to do it. (thispersondoesnotexist[.]com)

#usenix #DeepFakes AI models can synthesize hair from one person, glasses from another, pose from a third, skin tone from a fourth, and so on, to make a face with parameters that *you* choose. Need a white woman with long hair? AI can help you.

#usenix #DeepFakes Snapchat filters let you do this kind of image manipulation in real time for fun

and spies have used this to create a profile for a fake person. you can't reverse image search for someone who never exists apnews.com/bc2f19097a4c4f…

#usenix #DeepFakes You can synthesize audio of someone speaking: fakejoerogan[.]com

The speaker played two examples, one real and one fake, and then said "I swear to God I forget which is which."

#usenix #DeepFakes You can swap one person's face with another person's face *in a video* - a faceswap deep fake.

Presenter showed a video of a Tom Cruise impersonator, with Actual Tom Cruise's face swapped in.

#usenix #DeepFakes You can also do a lip-sync deep fake, like the Fake Zuckerberg vice.com/en_us/article/…

it's a real video of Zuckerberg, but with mouth movements changed to match the fake audio

#usenix #DeepFakes And there are "puppet master deep fakes" - you film an actor doing something and animate a fake face (or entire person) doing what the actor is doing

#usenix #DeepFakes Deep fakes have been weaponized for non-consensual pornography - that's where the name comes from (unfortunately).

They're used for misinformation, which is getting easier to make.

They're used for evidence tampering.

#usenix #DeepFakes There are national security implications to deep fakes, and child safety implications, and fraud concerns.

#usenix #DeepFakes So how do you detect a deep fake?

You can train a machine learning network to detect it. But a detector is built into deepfake synthesis - so there's a fundamental problem.

#usenix #DeepFakes Also, ML tends to latch on to relatively small details in a video. So downsizing and recompressing a video will probably get rid of the artifacts that an ML model would be able to detect as "fake".

#usenix #DeepFakes One thing to look for is "soft biometric properties" - the specific mannerisms that individual people have.

For example - [the speaker played a bunch of videos] - Obama tended to purse his lips between "Hi everybody" and the rest of the content of his speech.

#usenix #DeepFakes The current President, who people namesearch for, puckers his chin - but only when his mouth is closed.

Alec Baldwin's impersonations do the chin-puckering only when his mouth is open.

#usenix #DeepFakes Impersonators may not get all the quirks correct, and lip sync fakes decouple the person's face and their speech. (If a person always, or often, looks left when delivering bad news, a lipsync fake may not.)

#usenix #DeepFakes Face tracking technology is now good enough that you can you can feed even low-quality video to face tracking software and see how correlated pairs of features are. "Features" can be eyebrow movement, eyes looking left/right, expression, head orientation, etc.

#usenix #DeepFakes The speaker fed videos of several politicians (the current president, H. Clinton, Obama, Warren, Sanders) into a face tracker, and found that individuals were fairly consistent across time and different videos.

#usenix #DeepFakes Because individuals are fairly consistent in their speech quirks, you can build a Barack Obama Anomaly Detector from real videos of Obama.

#usenix #DeepFakes "We had 207,000 10-second clips of real Obama"

#usenix #DeepFakes "I'm not a big fan of AUC, but we have a DARPA grant, so we're stuck with it". a 50% AUC is no better than guessing, a 1.0 is perfect knowledge

#usenix #DeepFakes Their Obama Anomaly Classifier has an AUC of about 0.9 for determining whether someone is or is not Obama for videos of random people, videos of comedic impersonators, face-swaps, and puppetmaster fakes.

For lipsyncs it's about 0.85.

#usenix #DeepFakes The speaker guesses that the lipsync detection is harder because most of the video *really is* Obama.

if you do a majority vote over an entire video, sliced up into 10-second segments, you get an AUC above 0.9 for all the categories.

#usenix #DeepFakes If you focus on only a few features - mouth features and top-of-head features - you get a very good AUC.

#usenix #DeepFakes And this analysis is robust against "video laundering" (scaling a video down and recompressing it to reduce or change artifacts).

#usenix #DeepFakes "We're working our way through [models for detecting fake videos of] the Democratic candidates. We're hoping some of them drop out"

#usenix #DeepFakes For videos of H. Clinton, Sanders, the current president who people namesearch for, and Warren, models have at least a 0.9 AUC for detecting fakes.

Warren is very easy to detect fakes of because she's very expressive. "We love Elizabeth Warren" [laughter]

#usenix #DeepFakes In contrast, the current President who people namesearch for is much less expressive so fake videos are harder to detect.

#usenix #DeepFakes Obama behaves differently when he's giving a scripted speech and when he's speaking to someone one-on-one or without a script. So you need to build different models for that.

#usenix #DeepFakes All of this analysis only works on *video* - not individual frames, and not audio.

#usenix #DeepFakes However, audio has characteristics too! People have speech cadences. Obama tends to pause for about a second when he pauses. "Warren talks like a bullet train."

#usenix #DeepFakes You can also link audio cues to video cues. When Warren raises her voice, her eyebrows go up. That's an area for future work too.

#usenix #DeepFakes Audience Q: "It seems to me that you're doing the hard job, and watermarking the video is the easy job."

#usenix #DeepFakes Answer: "I'm about to say something where I have a financial conflict of interest" - he works for a company that does video authentication. In a perfect world, that would be sufficient. "But if it was a perfect world, I'd be out of a job."

#usenix #DeepFakes Audience Q: How easy is it to attack this?

answer: Well, deep fakes used to be detectable because they didn't blink. They blink now.

But we're looking at long segments, which generative models don't look at.

#usenix #DeepFakes it's difficult for models to generate the very long correlations across 10 seconds when they're working frame-by-frame.

And sure, a very sophisticated model could defeat this. But "some knucklehead on Reddit" couldn't.

#usenix #DeepFakes Audience Q: Is your research making deepfakes better (because fake-makers can see it)?

Answer: We're not making these models publicly available. We're publishing the paper, and we'll share our model with researchers, but we won't share it publicly.

#usenix #DeepFakes We're going to share our tool with journalists and factcheckers, but not publicly. I'm not sure that's the right solution but it seems like the best balance right now.

#usenix #DeepFakes Audience Q: How do you convince people that "this video of Barack Obama saying he'll stage a coup" is fake when your argument is "but his eyebrows move differently!"

A: I don't have a good answer there.

#usenix #DeepFakes We can say that with 95% probability this isn't a real video. That's part of a larger ecosystem of fact-checking. it can't be the only answer.

#usenix #deepfakes Audience member: "I have good news for you. Governor Hickenlooper just dropped out, so that's one fewer person for you to model." [laughter]

#usenix #deepfakes Q: How good an AUC is "good enough"?

A: We'd like to get to 99.9%. A 1-in-1000 chance is good enough for me.

#usenix #deepfakes Audience Q: what is the right way to think about using this kind of technology? Isn't this like microexpressions where it can change the way we socialize?

A: You can't just unleash this on journalists. We want to talk about training journalists.

#usenix #DeepFakes We want journalists to understand what an 80%-probability-of-fake means, and we want to help journalists understand that there is subtlety and complexity here. But it's hard and we're trying to figure this out.

#usenix #DeepFakes and now, we are going to drink non-machine-generated, non-fake, authentic coffee