This is a really important paper for #NLProc, #ethNLP and #ethicalAI folks.

The authors look deep into a use case for text that is ungrounded in either the world or any commitment what's being communicated but nonetheless fluent, apparently coherent, and of a specified style. You know, exactly #GPT3's specialty.

What's that use case? The kind of text needed, and apparently needed in quantity, for discussion boards whose purpose is recruitment and entrenchment in extremist ideologies.

And guess what? They find that #GPT3's trick of "few shot" training is definitely up to this challenge.

I don’t think GPT-3 could produce text written from the point of view of a conspiracy theorist if it didn’t have such texts among it’s training data. But, in the spirit of healthy skepticism, if someone wants to explain how it could, I’m curious about your theories. #NLProc

The next question then, is: how much such data does it need? Are we seeing a reflection of lots of this garbage getting sucked into the maw of the data-hungry algorithm? Or does it only take a little?

And if it only takes a little, that’s actually much worse, because it’s much harder to design processes that can filter out tiny amounts of this. E.g. would examples quoted in serious articles discussing the threat of online fora like this be enough?

My take away 1: ML systems that rely on datasets too large to actually examine are inherently unsafe. (Quote previous tweet on this.)

My take away 2: This paper shows the immense value of interdisciplinary perspectives in evaluating the potential risks of technology.

p.s. The paper goes talk about #GPT3 having "knowledge" of various conspiracy theories. I think this is a category error, but it does not detract from the point the paper is making. For more on why, though, see…

cc: @KrisMcguffie @AlexBNewhouse

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Emily M. Bender

Emily M. Bender Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @emilymbender

21 Jun
@robiupui I totally get that this is the most difficult option in many ways, but also I might be able to help because I have been teaching in this format since 2007, in order to accommodate both distance & local students in the MS program that I run. >>
@robiupui The problems to solve are:

1 Instructor audible to remote & local students
2 Slides visible to remote & local students
3 Instructor gestures visible to remote & local students
4 All students can ask questions/answer questions/comment
5 All students can hear student contributions
@robiupui 6 Remote students can turn in homework/take exams
7 Remote students can connect with out of class help (office hours, bulletin boards)
8 Remote students can collaborate with others (local or not).

6-8 are easy though and I imagine not what you're asking about. >>
Read 12 tweets
13 Dec 19
Tonight in briefly got tangled in a rather pointless Twitter argument about #AIethics—pointless because the other party isn’t interested in discussing in good faith. One point of clarity has come out of that though, in the responses of a few of us and I want to pull it out here:
Ethical considerations are not a separate concern from whether the AI research is “sound”: sound AI research requires not only valid math but also sensible task design.
A lot of the ethically questionable things we’re seeing (predicting faces from voices, predicting “intellectual ability” from short text responses, etc) are cases where it doesn’t make sense to predict A from B, by any model.
Read 9 tweets
23 Jul 19
Niven & Kao's upcoming #acl2019nlp paper "Probing Neural Network Comprehension of Natural Language Arguments" asks exactly the right question of unreasonable performance: "what has BERT learned about argument comprehension?"


They show, with careful experiments, that in fact, “BERT has learned nothing about argument comprehension.” Rather: “As our learners get stronger, controlling for spurious statistics becomes more important in order to have confidence in their apparent performance.” /2
This kind of careful work, featuring careful attention to the data, is exactly what #NLProc needs more of! /3
Read 4 tweets
19 Apr 19
I've seen several different #NLProc folks suggesting today that it would fun/interesting/worthwhile to use BERT or GPT-2 to fill in the redacted bits of the Mueller report. A short thread on why this is a terrible idea /1
First: consider the importance of the ability to find news sources that you trust and how much interest there is in the document. If you put out a version of that document with invented text in place of the redactions, how long before someone reposts it as the real thing? /2
How does that affect the discourse around what's actually contained in the (unredacted) version of the document, what it means, etc. both immediately and at some future point when the actual thing is available in full? How does it affect people's trust in reliable news? /3
Read 8 tweets
6 Mar 19
For the curious, here are the results of my #unscientificpoll about questions in double modals, plus some reflections:
So, "Might Pat could leave?" was slightly more popular than "Might could Pat leave?" but the majority answer by far was neither of those. >>
I suspect that's because many people who answered the poll just wanted to see the results, and 'neither' seemed like the best answer in that case. >>
Read 9 tweets
17 Jul 18
Currently* livetweeting: Anton van den Hengel’s keynote “Deep Neural Networks, and some things they’re not very good at"
(*provided @twitter doesn't lock me out again)
@Twitter van den Hengel:
My group works mainly on computer vision, grew from about 5 people in 2007 to 110 and growing now.
Read 100 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!