Thread by @LordLicoriceDrp

#aidungeon I wrote a summary on the AI Dungeon debacle elsewhere, so I figured I may as well share here. Bear with me, this is my first tweet thread like this.

Brief intro: AI Dungeon 2 (AID2) is a GPT-3 text adventure game. Enter a command and the AI builds the story. 1/

#aidungeon GPT-3 is powerful tech trained on ~800GB of stories sourced from the web. It's imperfect but generates believable, coherent narratives. It rolls with any crazy input you give it; "I fly to the moon", it'll describe the moon and populate it, etc. 2/

#aidungeon Remember how I said it's trained on stories from the web? That includes huge amounts of fanfic, including AO3 and Literotica: the-eye.eu/public/AI/pile…

So it turns out that this thing generates tailored smut really well in addition to adventuring. Neat! Well... 3/

#aidungeon In a story where you can do ANYTHING without consequence, some obviously write really kinky, dark stories, including loli content (sometimes unintentionally but we'll get to that). The devs were forced to reckon with this element.

So they silently added a filter. 4/

#aidungeon This turned out to be a disaster. First, the filter is *very* stupid; given the complexity of English, it was banning *any* reference to child/girl/boy, numbers under 18, pets/animals, siblings, and (allegedly) Black and Asian people. My favorites so far: 5/

#aidungeon So censorship sucks, but oh well, right? Well, no: If your story is flagged, the @AiDungeon team combs through your private stories looking to ban you. Imagine iCloud flagging your beach photo and sending an intern to browse your private galleries for CP. 6/

#aidungeon This is in direct contradiction to their previous privacy statements: without notice, they reversed policy, and now explicitly review your content manually if this crappy filter marks you. Even if you don't use it for smut, this is a significant invasion of privacy. 7/

#aidungeon Back to censorship: Remember how I said it was for loli content but it's banned siblings, pets, and Black people? It seems they also filter for incest, bestiality, and racism. The devout Mormon dev team is pushing their values onto the userbase; LGBT may be next. 8/

#aidungeon "Well maybe people shouldn't use it for porn then!" First, a significant portion of the paying userbase writes smut. More importantly, the dev team chose to use smut in the training data; this use of the tech was a foregone conclusion. 9/

#aidungeon Wait, how do I know what the userbase does?

BECAUSE AI DUNGEON SUFFERED A MASSIVE DATA BREACH, WHICH THE COMPANY HAS YET TO ADDRESS. They repeatedly failed to address known a leak, so someone scraped it for ALL STORIES WITH USERNAMES: 10/
github.com/AetherDevSecOp…

#aidungeon So we've got free speech absolutists protecting ALL writing, innocents and smut-lovers with interrupted games, untrained/underaged Discord mods contradicting themselves/policy, a massive data breach, and total radio silence from Latitude. In short: a shitshow. 11/

#aidungeon Oh, and the kicker of it all? The filter is on user *input*; the outputs are thus far unaffected. A big problem of AID2 is it forgets characters' descriptions... including age. Also, it can abruptly spit out some ludicrous, unexpected story beats, like this: 12/

#aidungeon You can enable SFW to filter sexual content fairly well, but NSFW stories can unexpectedly veer into uncomfortable scenarios. This problem is intractable; even SFW stories tend toward romance due to the fanfic in the training data, a common complaint. 13/

#aidungeon Unlike some bots, the AI *cannot learn further* from user entries; it only knows what sex is because it learned in the original dataset. The devs brought this situation on themselves by choosing that dataset, and are making ugly decisions to sloppily fix it. 14/

#aidungeon So where does that leave us? Well, the userbase is pissed and unsubbing in droves, for a variety of reasons, and the devs are doubling down and accusing the userbase of supporting pedophilia rather than address the actual issues. They haven't yet ack'd the breach. 15/

#aidungeon My personal takes: I could put up with a filter, but the invasion of privacy is unconscionable. While I'm firmly in the "fiction is fiction" camp, and I understand what the devs are *trying* to do, I don't agree with it nor with their implementation. 16/

#aidungeon I don't know if this is the end of the game, and I feel bad for Nick Walton, who clearly never wanted any of this to happen. Nevertheless, Latitude made this mess, and they need to acknowledge the ACTUAL problems if they want to stay solvent. 17/17

"This isn't a child having their toys taken away nearly so much as it is suddenly revealing there was a hidden camera in the bedroom all along." vice.com/en/article/93y…