Thread by @informor

🚨New Pub at PNAS🚨! We know people cannot detect language written by AI. But what makes them THINK text was AI-generated? We show that people have consistent heuristics... that are flawed & can be exploited by AI to create text "more human than human" 🧵

pnas.org/doi/10.1073/pn…

In the first part of this work (lead author: @maurice_jks) we collected 1000s of human-written self-presentations in important contexts (dating, freelance, hospitality); created (#GPT) 1000s of AI-generated profiles; and asked 1000s of people to distinguish between them. 2/

They couldn't (success rate: ~50%). However, they were consistent: people had specific ideas about which profile was AI/human. We used mixed methods to uncover these heuristics (next tweet) & computationally show that they are indeed predictive of people's evaluations... 3/

... but rarely predictive of whether the text was ACTUALLY AI-generated or human-written. The figure below shows some of the features people associated with AI (network icon) or human (human), and whether this heuristic was generally wrong (red), or correct (green). 4/

For example, use of rare bigrams & long words was associated with people thinking the profile was generated by AI. In reality, such profiles were more likely to be human-written. Use of informal speech and mentions of family were (wrongly) associated with human-written text 5/

To validate these findings, we ran an experiment. Since there are predictable features that make people think a profile sounds more "human", AI can take advantage of these features/human heuristics to create profiles that are "more human than human". Or can it? 6/

Our experiment asked for human/AI labels for profiles but this time included "optimized" profiles, predicted to be rated as more human. Indeed, across contexts, people rated "optimized" profiles MUCH more often as human, compared to the human or "regular" AI-generated profiles 7/

Why is this important? It's now clear that more of our online content and communication will be generated by AI. In our previous work, we demonstrated the "Replicant Effect": as soon as the use of AI is suspected, evaluations of trustworthiness drop. 8/

dl.acm.org/doi/10.1145/32…

In the current work, we show that not only people cannot distinguish between AI and human-written text, but that they have heuristics that can be exploited by AI, potentially leaving the poor authentic humans to suffer decreased trustworthiness evaluations. 9/

As we rush into deploying and using large language models and applications like #ChatGPT in our everyday communication, our paper asked to stop and consider the consequences as we move from Computer-mediated to AI-mediated communications (AI-MC). /10

pnas.org/doi/10.1073/pn…

The paper was a @cornell_tech and @Stanford collaboration led by @maurice_jks (🚨on the job market in Europe🚨) and with @jeffhancock. I will write about the origin of this NSF-funded research next. Also, more exciting AI-MC results to come soon! /end

I neglected to add the (slightly older and not final but includes all the results/data) open-access version:

arxiv.org/abs/2206.07271