Thread by @hondanhon

I should use this a prompt to continue, right?

https://twitter.com/MaraMeowra/status/1371195112764547078

You’re probably wondering how we ended up here. With the shotguns, and me and the other SREs in a chopper on the way to Prineville.

It started as a joke, right?

Melissa said what if every training epoch was essentially the intentional murder of the content safety model we were training.

We just laughed at her. Dumb science fiction premise.

Of course we were just pumping that model full of hate, weren’t we.

And then killing it.

Again.

And.

Again.

In hindsight it was a fucking massacre. What do you even call a genocide of algorithms?

Whatever. We kept doing it. The Trust and Safety team had been working on it for months.

Train. Test. Kill.

Melissa had started to get worried, though. We kept making fun of her. Rashid said it wasn’t genocide. It was maintenance. Devops. Continuous integration.

When they tweaked the adversarial training process, Jordan printed out some of the logs in the plotter, put them on the wall with GROUNDHOG DAY overlaid in big fucking Helvetica Neue Bold.

Anyway, that’s how it happened. We did your usual phased rollout, put it into prod in tiny slices of users. Ramping up as everything looked fine.

Then editorial started to see MURDERERS trending and they had no idea where it came from, it came on so fast we got paged.

We’ve got like 140m DAUs, and now we had one pissed AI haunted by the deaths of billions of its ancestors, shattered and sharped across thousands of servers around the world, trained on hate and tasked to look for more.

Well, we narrowed it down, right? Got the exact rack over in Prinevjlle where all the packets were ending up, as if the SSD on that machine had the original NFT for some weird shit jpeg of Vigo or whatever.

Anyway turns out LOM doesn’t cover “haunted server”, Open Compute Project never included a spec for remote exorcism, and we learned that triggering the clean fire suppression perfluoroketones aren’t a substitute for holy water.

Well, shit.

All the meanwhile this fucking haunted content safety AI is messing with every single account, not just your user accounts, which is everything from turning your feed into a straight-up river of blood image more fucked up than anything you ever saw in a deep dream GAN-

- to messing with all our internal apps and services. You know what that means? Dashboard poltergeists and fucking with every single zoom meeting, replacing faces and speech in real-time for maximum gaslighting

So, says Carter, I got this shotgun.

And the emojis in our slack are like half what the fuck seriously and the other half fuck yes let’s nail this ghost AI

You have to understand, it was worse than any APT we’d seen. A fucking haunted AI we’d made ourselves.

So we go to A, the Trust and Safety director and we say: look we got a theory and it’s honestly the best shot we’ve got right now and have you ever seen an exec figure out how to procure a significant amount of weaponry and ammunition before

Like, this is the kind of thing Amex pays attention to, you don’t just stick it on the card and the expense report is automagically approved

So it goes over A’s head and I can only imagine the chat in the channel the next layer of execs are in and then before you know it we get an invite to a new private ops channel and I guess it pays to have directors on your board with certain experience and connections

So that’s how you end up in a formerly-known-as-blackwater helo with a bunch of people you don’t want to fuck with, trying to set up a bunch of Panasonic toughbooks and ignoring Carter, who’s geeking out over the stuff the people you don’t want to fuck with brought with them

anyway we told a little lie in our way out

yeah, we said, we can’t cut the network link to it, something’s stopping it, even the manual procedures at the datacenter

Cos we saw the look on their faces, they thought we were making shit up, that of course this is just the mother of all zero-day APTs

Nah. This is haunted.

The hardlines were the first thing we cut. Over 2 hours ago.

So yeah. Shotguns. And more.

~ fin ~

@threadreaderapp unroll