Thread by @CriticalAI

In the effort curb misunderstanding and #AIHype on the topic of language models (LMs), we're circulating a tweet thread to offer a baseline understanding of how systems such as OpenAI's GPT-3 work to deliver sequences of human-like text in response to prompts. /1

We're directing this thread especially to our humanist readers who may be only peripherally aware of the increased commercialization of (and hype about) "artificial intelligence" (AI) text generators.
NB: AI is itself a slippery term: we use it w/ caution.
/2

The best known of these models is OpenAI’s GPT-3, which is licensed by Microsoft. Students can use them to generate human-like text by paying OpenAI directly or subscribing to subsidiary “apps." They may also access less powerful but free models for text generation. /3

About one year into the release of GPT-3, teachers are circulating anecdotal accounts of auto-generated writing of passing quality or better. Students are claiming to have successfully used LLMs to produce assignments or parts of assignments. /4

Such anecdotes are often accompanied by loose talk of these models "disrupting" education. That kind of meaningless business-speak feeds into unnecessary hype and anxiety. A better course is to educate oneself and one's students about the affordances & limitations of LMs. /5

Exacerbating the hype, many journalists are struggle to distinguish between marketing and reality in describing these technologies to the general public; @nytimes, @guardian, @TheAtlantic have fallen into this trap. /5

So let's be clear: LMs are sophisticated statistical models that predict probable sequences of language in response to a prompt even though they do not “understand” language in any human-like sense. /5

Instead, through intensive mining, modeling, and memorization of vast stores of language data “scraped” from the internet, LLMs deliver word sequences that, from the standpoint of human readers, LOOK like text authored by other humans. /6

Bear in mind that humans are highly susceptible to projecting human status onto any entity that appears to be generating human language. This tendency to anthropomorphize has been documented since the 1960s and is called the Eliza Effect. /7

By design, the synthetic text LMs generate isn't directly “plagiarized” from some original, and it is usually grammatically and syntactically well-crafted. From an academic integrity perspective (for those teaching students), this means that AI-generated writing.../8

1) is not easily identifiable as such to the unpracticed eye; 2) does not conform to “plagiarism” as that term is typically understood; AND 3) encourages students to think of writing as task-specific labor disconnected from learning and the application of critical thinking. /9

Regrettably, some academics have begun to generate the misleading idea that statistically generated writing poses an existential dilemma for the humanities--that it's set to replace teachers, humanities courses, and creative writing. This is ungrounded #AIHype /10

This lamentable hype leads ppl to believe that LMs' textual outputs are objective, reliable, and accurate. In reality, LLMs are riddled with falsehoods, misconceptions, and bias which they mimic from training data “scraped” from social media platforms. /11

The ground breaking 2021 article on "Stochastic Parrots" by @emilymbender, @timnitGebru, @mmitchell_ai et al. documented many of the harms of these models. /12 dl.acm.org/doi/10.1145/34…

More recent research has confirmed their findings along multiple fronts. LMs have been found to generate conspiracy theories and climate change denial [see Lin et al. 2022; Evans et al. 2022] /13

@athundt and colleagues found that robots trained on an OpenAI's CLIP leads robots to act out "toxic stereotypes with respect to gender, race, and scientifically-discredited physiognomy, at scale" dl.acm.org/doi/10.1145/35…

@athundt Whereas hype leads ppl to believe that LMs generate meaningful communication (critical "thinking,” or even “sentience”). in reality "stochastic parrots" model language SEQUENCES, not the MEANING of language./15

@athundt Unlike a human w/ something to say, LMs generate multiple outputs upon a user’s request including outputs that are fabricated, false, racist, misogynistic or nonsensical. They are no more “sentient” or “conscious” than a digital assistant. /16

For experienced writers, alert to the systems' many pitfalls, and knowledgeable about how to edit prose, playing with LMs can be fun. But their ability to mimic but not to perform critical thinking makes them a poor & even dangerous helpmate for many students. /17

For example, GPT-3 attributes quotations to famous authors that they did not write; fabricates non-existent citations that look like the real thing. They will write a short academic bio for you that includes jobs and degrees that you never had. /18

Alert to these issues, @PaulaKrebs (exec director of MLA) and @cnewf are working w/ counterparts in CCCC to convene a task force to help teachers teach themselves & their students about LMs and their risks /19

@PaulaKrebs @cnewf #CriticalAI supports this collaborative endeavor. In the meantime we're happy to share ideas for essay prompts that make it impossible for students to rely on text generators to perform the critical thinking that goes into their written work--the most important part! /20

As those reading these tweets know well, #CriticalAI is about to launch a new interdisciplinary journal with @DukePress. The journal is intended to build collaboration between humanists, interpretive social scientists, and technologists over big questions like these. /21

Our approach to "AI" is community- and public interest-centered. Inspired by @schock and others who cultivate and enact Design Justice principles, we seek technologies that "prioritize design’s impact on the community over the intentions of the designer."/22

@schock We're v. excited to partner with @DAIRInstitute in a new "#AI Hype Wall of Shame" (coming soon)--through which we hope to collectivize the effort to educate the public, and support journalists who take "AI "seriously" w/o regurgitating talking points & marketing campaigns. /23

In what's below we offer a small sample of expert resources that you might wish to consult, use in your teaching, or recommend to your students beginning with the @AINowInstitute's brilliant lexicon medium.com/a-new-ai-lexic…

@schock @DAIRInstitute @emilymbender is ofc a formidable counter to #AIHype. Here for example is her detailed and crucial riposte to an NYT article from this summer which hyped GPT-3's alleged "mastery" of language /25 medium.com/@emilymenonben…

@schock @DAIRInstitute @GaryMarcus's co-authored review of GPT-3 in the MIT Review is "must reading"--as his ongoing substack which we follow regularly. technologyreview.com/2020/08/22/100…

For strong thinking on policy, we recommend @FrankPasquale's book, New Laws of Robotics: Defending Human Expertise in the Age of AI (2020)/26

We are longstanding admirers of
@safiyanoble
, whose
@C2i2_UCLA is one of our favorite partners. Many of you know that her Algorithms of Oppression is indispensable reading for an understanding of the built-in bias in search and much else. /27

@safiyanoble @C2i2_UCLA @merbroussard has a new book coming out soon but her ARTIFICIAL UNINTELLIGENCE is still one of the best introductions to machine learning! mitpress.mit.edu/9780262537018/…

We also embrace the Digital Humanities world. @KatherineBode is the lead organizer of our ongoing #NEH-sponsored collaboration with Australian National University. Last year we read the work @laurenfklein and @kanarinka--among many others in our workshop on data curation. /29

The same workshop series included this great post from @mmvty who leads the @PrincetonDH criticalai.org/2022/04/13/dat… /30

And on the matter of language models, we also recommend the videos from computer scientist Matthew Stone and @kathbode embedded in this post: all on "Stochastic Parrots" criticalai.org/2021/10/14/blo… /31

If you're still reading, we hope that you will follow us and share your ideas w/ us via this account or that of our many partners. Here's the Editorial Collective (our Advisory Board is still in formation!). criticalai.org/steering-commi…

Please think of #CriticalAI as an open door. We're keen to read some of your work, learn more about your research, get your feedback on the journal (or blog or the Wall of Shame), find out what you're reading & what books or special issues you'd welcome. /33

If you have any questions on language models specifically either DM us or write to us at [e-mail]
It seems fitting for this thread to leave the parting words the P.E. /end