Thread by @Helenreflects

Remember Cambridge Analytica? Not long ago right?
So I wonder why my friends are enthusiastically sharing aged/gender swapped pictures of themselves on an app called #FaceApp
The creator is a Russian tech firm we don't know much about, Wireless Lab. 1/
forbes.com/sites/haroldst…

While you are enthusiastically aging yourself it's important to note that the technology behind this is a deep neural network which learns from the data it gets. Think of AlphaGo, AlphaZero, and how scarily good AlphaGo got just by training itself on amateur games. 2/

AlphaGo was able to defeat the world Go champion. Note that before the use of these neural networks, computers weren't able to beat a good amateur. But by training themselves they can defeat the world champion, see here en.wikipedia.org/wiki/AlphaGo_v… (also cool Netflix docu) 3/

So now you have similar technology but this time it trains itself on lots and lots of human faces. The results are so much better than photoshop because it is able to correct itself, learn from its mistakes, by the massive n of pics we feed it 4/

Now you may wonder: what about my privacy? Well FaceApp states it can do whatever it wants with what you put into it. "You grant FaceApp a perpetual, irrevocable, nonexclusive, royalty-free, worldwide, fully-paid, transferable sub-licensable license..." 5/

" to use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, publicly perform and display your User Content"
Hello, irrevocable? Reproduce and modify? It gets worse... 6/
faceapp.com/terms

They say they won't sell your data to 3rd party, but if they are sold (to an entity that you'd rather not have access to your data) they can sell your user content along: "If we sell or otherwise transfer part or the whole of FaceApp or our assets to another organization... 7/

"... your information such as User Content and any other information collected through the Service may be among the items sold or transferred." 8/
faceapp.com/privacy?fbclid…

Ok scary perhaps but what would a face recognition neural network AI software be doing with your face? Well, they may be creating deepfakes, videos impossible to distinguish from real (see here by @rinireg nytimes.com/2019/06/10/opi…) 9/

And this is worrisome since we won't be able to tell what's real and what isn't as the by then well trained AI will be able to manipulate, say, politicians in saying something they did not in fact say, and we won't be able to tell the difference. 10/

Here's my worry about the broader implications for democracy. Cambridge Analytica used data from people filling out quizzes for fun, and their friend network. This system is much more sophisticated. 11/

You may recall this excellent article by @carolecadwalla on how massive misinformation was selectively directed at people who were susceptible to e.g., scare stories about immigrants "flooding"the country or being criminals in runup to Brexit 12/
theguardian.com/technology/201…

The far right (led by pluto-populists who divert people's attention away from their wealth, to scapegoats) is relying on massive disinformation to stoke up xenophobia, Islamophobia etc. Disinformation so rampant it's called "epistemic rights violation" 13/ tandfonline.com/doi/abs/10.108…

Imagine how this disinformation will affect voters once deepfake videos become available. And with sophisticated AI training itself on thousands of user-supplied content, it is going to happen. /end

A few replies to objections I've seen on this thread:
Objection 1: But already our images are all over the internet! What more does it do? Well the aim of this program is to train a neural network so it can manipulate faces. PS1/

It will collect both your face (nicely centered and uploaded by you) + information and information about your browsing history, see "These tools collect information sent by your device or our Service, including the web pages you visit, add-ons"... faceapp.com/privacy PS2/

So the neural network (we don't know how the NN operates!) is trained both to recognize faces and can link these to browsing behaviors of people, which is worrisome indeed. PS3/

Objection 2: Twitter and other platforms also have not so good privacy policies. True but this one is still pretty extensive. Also while I get some substantial informational etc benefit from Twitter, the same is not true of this app. It's giving them info for little return PS4/

Objection 3: "The Neural Network is not using your millions of faces to train itself". Well we know it does learn.
Compare "female" Steve Jobs done with the app in 2017 to the current version in 2019. The current version is MUCH BETTER. My speculation: the app learns. PS5/

So how does it learn? My speculation: user feedback. It can do this by gauging whether a pic is being shared via the app. If it is, it's more likely to look "convincing" than a pic not shared. With millions of people doing this, the app can learn what looks convincing. PS6/