Thread by @emilymbender

Okay, I read it so you don't have to. Here's a reaction thread to @openAI / @sama 's blog post from Friday "Planning for AGI and beyond":

openai.com/blog/planning-…

@OpenAI @sama From the get-go this is just gross. They think they are really in the business of developing/shaping "AGI". And they think they are positioned to decide what "benefits all of humanity".

Then @sama invites the reader to imagine that AGI ("if succesfully created") is literally magic. Also, What does "turbocharging the economy" mean, if there is already abundance? More $$$ for the super rich, has to be.

@sama Also, note the rhetorical sleight of hand there. Paragraph 1 has AGI as a hypothetical ("if successfully created") but by para 2 it already is something that "has potential".

But oh noes -- the magical imagined AGI also has downsides! But it is so so tempting and important to create, that we can't not create it. Note the next rhetorical sleight of hand here. Now AGI is an unpreventable future.

What's in fn1? A massive presupposition failure: The GPTs are learning information about word distributions in lots and lots of text + what word patterns are associated with higher scores (from human raters). That's it.

Then a series of principles for how to ensure that AGI is "beneficial". This includes "governance of AGI" as something that is "widely and fairly shared", but I've seen exactly nothing from @OpenAI about or advocating for building shared governance structures.

@OpenAI Meanwhile, "continuously learn and adapt by deploying less powerful versions of the technology" suggests that they think that the various GPTs are "less powerful versions of AGI".

<recordscratch> hang on: did he just say "maximarlly flourish in the universe"? What kind of weirdo longtermist, space-colonizing fantasy is that coming from?

Similarly here, this seems designed to promote the idea that the models they have already put into their API (GPT-2, GPT-3, ChatGPT) are the early stages of "AGI" being "stewarded into existence".

Then there's a glib paragraph about how "most expert predictions have been wrong so far" ending in footnote 2:

Paraphrasing: "Our experts thought we could do this as a non-profit, but then we realized we wanted MOAR MONEY. Also we thought we should just do everything open source but then we decided nah. Also, can't be bothered to even document the systems or datasets."

Hey @OpenAI, I'm speaking to you from 2018 to say: DOCUMENT YOUR DAMN DATASETS. Also, to everyone else: If you don't know what's in it, don't use it.

Source: aclanthology.org/Q18-1041.pdf

@OpenAI Okay, back to @sama. "As our systems get closer to AGI" -- here's a false presupposition again. Your system isn't AGI, it isn't a step towards AGI, and yet you're dropping that in as if the reader is just suppose to nod along.

Oh, and did you all catch that shout out to xrisk? Weirdo longertermist fantasy indeed.

As I said in my thread yesterday, I wish I could just laugh at these people, but unfortunately they are attempting (and I think succeeding) to engage the discussion about regulation of so-called AI systems.

What's needed is regulation about: how data can be collected and used, transparency of datasets, models and the deployment of text/image generation systems, recourse and contestability of any automated decision making, etc.

Talking about text synthesis machines as if they were "AI" muddies the waters and hampers effective discussions about data rights, transparency, protection from automated decision systems, surveillance, and all the rest of the pressing issues.

The problem isn't regulating "AI" or future "AGI". It's protecting individuals from corporate and government overreach using "AI" to cut costs and or deflect accountability.

The contradiction in these next 2 paras is stunning: We think you should be able to do whatever you want with our systems, bc "diversity of ideas" but also we think we can align the systems with "human values". So, assholes can create fake revenge porn, but that's okay because-?

LOLOL -- calling something a "ratio" doesn't make measurable or, ahem, real.

[This is exhausting, but I started it. Might as well finish.]

Wait what -- now they're talking seriously about "late-stage AGI development"?

Here's a bunch of promises about future oversight by unnamed independent auditors and also "major world governments" (who counts as major? who decides?). Also, how about just DOCUMENTING YOUR DAMN DATA for everyone to see?

"Continuum of intelligence" is gross, not least for the suggestions of ableism, eugenics, transhumanism etc. But also "rate of progress [of] the past decade" -?Progress towards what? Ever larger carbon footprints? More plausible fake text?

And, more to the pt: There are harms NOW: to privacy, theft of creative output, harms to our information ecosystems, and harms from the scaled reproduction of biases. An org that cared about "benefitting humanity" wouldn't be developing/disseminating tech that does those things.

No, they don't want to address actual problems in the actual world (which would require ceding power). They want to believe themselves gods who can not only create a "superintelligence" but have the beneficence to do so in a way that is "aligned" with humanity.

/fin