Thread by @conspirator0

We've done a decent amount of research on the use of GAN-generated images over the last two years, mostly fake face "photos" such as those produced by thispersondoesnotexist(dot)com. Here are all of our related threads in one place.

cc: @ZellaQuixote

GAN is an abbreviation for "generative adversarial network", the AI technique used to produce these images (and other things). Here's a "brief" description of how StyleGAN, the GAN behind thispersondoesnotexist(dot)com, works.
machinelearningmastery.com/introduction-t…

Here's the first network we found in the wild using GAN-generated face pics (29 of 52 accounts, created in two batches). The profiles featured repetitive biographies mentioning family, friends, country, and weapons.

Next up, a coronavirus/bitcoin-themed botnet from March 2020. 16 of 28 accounts used GAN-generated face pics.

In May 2020, we ran into a set of five fake pro-Biden accounts, all using the same GAN-generated profile pic. These accounts self-deleted shortly after this thread was posted.

Our first stab at a programmatic method for detecting GAN-generated face pics. This technique is designed to make the GAN-generated pictures easier to notice in groups of thousands of images - it isn't terribly accurate for individual images.

From May 2020, a 41 account Spanish-language botnet using GAN-generated profile images, with a focus on Argentina.

Next, a 4chan op using accounts with GAN-generated pics to push the #BernReturn hashtag alongside a bogus claim that Bernie Sanders was returning campaign donations.

Speaking of 4chan, thispersondoesnotexist(dot)com is quite popular over there, and folks frequently suggest using it as a source of profile pics for bogus Twitter accounts.

Here's a thread on now-suspended part-automated political troll account @Juan6million. This account mostly pushed left-wing messaging and hashtags, but also promoted alt-right talking points and influencers such as @MrAndyNgo.

The most prolific use of GAN-generated profile pics we've seen thus far was from the "Thousand Followers" follower-buying website (presently on hiatus). 8727 of 17957 of the fake Twitter followers provided by this site used GAN-generated images.

From August 2020, a mixed commerical/political retweet botnet consisting of 21 accounts using GAN-generated face pics.

We've run across a couple of botnets promoting cryptocurrency sites that use GAN-generated profile pics: 41 accounts pushing cointelegraph(dot)com and 28 accounts pushing ethereumcryptocurrency(dot)com.

Deepfake human faces are not the only sort of images one can produce with GANs. Here's a thread on GAN-generated anime pics, and the detection thereof:

From November 2020, a fake follower/retweet botnet using GAN-generated profile pics (164 accounts).

A similar network follower/retweet botnet created in August 2020. 53 accounts, all using GAN-generated pics. It mostly amplifies Russian-language accounts.

Thread on a weird automated account (@Jtatejtate1) using a GAN-generated face. A bunch of its followers are part of a recently reactivated botnet from 2013.

Using reverse image search to find Twitter accounts using GAN-generated profile pics: