Thread by @zilevandamme

Deepfake researchers have long feared the day this would arrive - an app breaching an ethical line no other service has crossed before.

technologyreview.com/2021/09/13/103…

"MIT Technology Review has chosen not to name the service, which we will call Y, or use any direct quotes and screenshots of its contents, to avoid driving traffic to the site. It was discovered and brought to our attention by deepfake researcher Henry Ajder."

"From the beginning, deepfakes, or AI-generated synthetic media, have primarily been used to create pornographic representations of women, who often find this psychologically devastating."

"Many of these services have since been forced offline, but the code still exists in open-source repositories and has continued to resurface in new forms. The latest such site received over 6.7 million visits in August... It has yet to be taken offline."

"But as the first dedicated pornographic face-swapping app, Y takes this to a new level. It’s “tailor-made” to create pornographic images of people without their consent, says Adam Dodge, the founder of EndTAB, a nonprofit that educates people about technology-enabled abuse."

"Many of the face swaps are obviously fake...But to a casual observer, some are subtle enough to pass, and the trajectory of deepfakes has already shown how quickly they can become indistinguishable from reality."

"Some experts argue that the quality of the deepfake doesn’t really matter because the psychological toll on victims can be the same either way. And many members of the public remain unaware that such tech exists, so even low-quality face swaps can be capable of fooling people."

"The consequences for women and girls targeted by such activity can be crushing. At a psychological level, these videos can feel as violating as revenge porn—real intimate videos filmed or released without consent."

"And the repercussions can stay with victims for life. The images and videos are difficult to remove from the internet, and new material can be created at any time."

“It affects you with getting jobs. Every single job interview you ever go for, this might be brought up. Potential romantic relationships. To this day, I’ve never been successful fully in getting any of the images taken down. Forever, that will be out there. No matter what I do.”

For women in South Africa. A revenge porn law exists. It does need to be expanded, but a court of law would interpret it generously. I have seen many memes created, and there are no doubt deepfake videos. In terms of the The Films and Publications Act revenge porn is a crime.

The @FPB_ZA website has been offline for weeks, suspended. A state entity meant to deal with online violence. I mean. But the law exists and can be enforced. I would like to see the first case where the revenge porn law is tested.

(FPB pay your website host, goodness!)

The Cybercrimes Act could also apply, specifically s16
gov.za/documents/cybe…

But in a nutshell, legislation is needed to be quite specific dealing with online violence against women, including AI, not just revenge porn. Don't give me "free speech," you don't have free speech to create sexualised images of women online and emotionally abuse them.

I seem to have broken the thread, it continues here:

https://twitter.com/zilevandamme/status/1437366920148307971