Profile picture
Sam Gregory @SamGregory
, 25 tweets, 10 min read Read on Twitter
1/ What are #deepfakes + #AI generated synthetic media; what threats do they pose to #humanrights, #journalism and #publicsphere? What should we fear? What can we do? Thread w/ key tech + usages from recent expert mtg summarizing…
2/We've had fake images and CGI for years, so what's different now (beyond enabling us to transplant Nicolas Cage's face to other people). The ease of making #deepfakes is captured by the vlogger Tom Scott
3/ = Barriers to entry to manipulate audio and video are falling - costs less, needs less tech expertise, uses open-source code + cloud computing power. Plus, sophistication of manipulation of social media spaces by bad actors makes it easier to weaponize and to micro-target.
4/Leads to increasing possibility of REALITY EDITS: removing or adding into photos and videos in a way that challenges our ability to document reality and preserve the evidentiary value of images, and enhances the ability of perpetrators to challenge the truth of rights abuses.
5/More ability to create CREDIBLE DOPPELGANGERS of real people that enhance the ability to manipulate public or individuals to commit rights abuses or to incite violence or conflict. Could be Trump, Aung San Suu Kyi or your local politician.
6/Greater potential for NEWS REMIXING that exploits peripheral cues of credibility and the rapid news cycle to disrupt and change public narratives. For a pre-deepfakes e.g. see this recently from Moldova:…
7/PLAUSIBLE DENIABILITY for perpetrators to reflexively claim “That’s a deepfake” around incriminating footage or taken further, to dismiss any contested information as another form of fake news... 'It's all fake news, don't believe anything you see'
8/FLOODS OF FALSEHOOD created via computational propaganda and individualized micro-targeting, contribute to disrupting the remaining public sphere and to overwhelming fact-finding and verification approaches.…
9/ It matters to push back vs this - @witnessorg we know critical value of + people able to share videos/ images 2 expose injustice, share powerful civic storytelling, mobilize movements. We need to defend that and fight evolving probs of image manipulation + image-based violence
10/ So we convened an expert meeting to get proactive on solutions around #deepfakes and #AI generated media. You can read the full-report out here:…
11/ First question. What are talking about here? What are new and improved tools for audio and video manipulation and creation (often based on #deeplearning, #GANs and other ML approaches) that have malicious uses?
12/INDIVIDUALIZED SIMULATED AUDIO: Enhanced ability to simulate individuals’ voices as developed and available commercially via providers like @Lyrebird or Baidu DeepVoice. Can be used for good - e.g. and for creativity, but also 4 malicious impersonation
13/ Emerging consumer tools that make it easier to SELECTIVELY EDIT, DELETE OR CHANGE FOREGROUND OR BACKGROUND elements in video. Concepts such as Adobe Cloak advance image editing currently available in tools like Photoshop or Premiere for +seamless editing of elements in video.
14/ FACIAL RE-ENACTMENT: Using images of real people as “puppets” and manipulating faces, expressions and upper body movements. Face2Face, Deep Video Portraits allow transfer of facial/upper body movements of 1 person to another person’s face/upper body.…
15/ Realistic FACIAL RECONSTRUCTION AND LIPSYNC created around existing audio tracks of a person as seen for example with the LipSync Obama project.…
16/ Real people with EXCHANGE OF ONE REGION, TYPICALLY FACE: As with deepfakes created via FakeApp or FaceSwap; also relates to tech in tools like Snapchat with a simulation of the face of 1 person imposed over the face of another person or in which a hybrid face is produced
17/ Combinations such as a DEEPFAKE MATCHED WITH AUDIO (FAKE OR REAL) AND RETOUCHING , e.g. the Obama-Jordan Peele video in which the actor-director Jordan Peele made a realistic Obama say words that Peele himself was saying.
18/ We're now basically entering arms race between manual/automatic synthesis of media, vs. manual and automatic forensic/detection approaches. Manual synthesis includes much of the innovation in CGI in recent years; automatic synthesis includes many of the tools shared here
19/Manual forensics is rich field of practice drawing on computer vision/signal processing - e.g. includes explicit checks of perspectival geometry, lighting, shadows and ‘physics’ of images, + detecting copying and splicing btw images or evidence of camera model for photo
20/ Automatic forensics often builds on similar tools as 4 creation of fakes incl. neural networks, #deeplearning, ML. Including detect copy splice or different camera models; heat maps of fake pixels from deepfake; identifying background sources of elements of generated images
21/ + Use of neural networks to detect physiological inconsistencies in synthetic media, e.g absence of blinking ( or use of GANs to detect fake images based on training data of images created using existing tools e.g. FaceForensics
22/ BUT automatics systems are trained on specific databases, and might detect mainly the inconsistencies of specific synthesis techniques, and OH DEAR... counter-forensics can use same tools (e.g. GANs) to wipe or fake forensic traces of cameras
23/Researchers disagree if “arms race” will be won by forgers or detectors. Humans are not good at detecting difference btw real and fake but w/ sufficient training data could automatic detectors keep up with generators? We'll see... Next thread on solutions/recommendations.
Your own prognosis on whether detection will keep up with fakery? (noting that detection of course is only the start of addressing the malicious use)
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Sam Gregory
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!