I reported on an experiment this week that blew my mind. Psychologists at @Cornell recruited thousands of people to talk with ChatGPT about a conspiracy theory they believed. They wanted to know: Is it true that conspiracy theories rarely get convinced out of their beliefs? 🧵
JFK, 9/11, the 2020 election being stolen, Walt Disney being cryogenically frozen -- participants in the study were asked to explain why they believed the conspiracy theory, and to rate their belief from 1 to 100. (100 being total certainty.)
The researchers then fed each participants answers into a ChatGPT model, and told it to use strong facts and evidence to debunk the conspiracy theory. They called it Debunkbot.
The interaction was limited to just three back-and-forths – three responses from the chatbot, three reactions from the participants. Then people were asked to rate their belief in the conspiracy again.
The results were stunning.
About a quarter of people who believed a conspiracy theory at the start of the conversation with the chatbot no longer believed it was true at the end. A quarter!
Just think about that: an 8-minute conversation with an AI model, and someone who believed with total certainty that the U.S. government killed thousands of Americans by demolishing the World Trade Center and then covering it up, could come away realizing that wasn’t true.
“We were like, there's a mistake. This is messed up," @GordPennycook, one of the researchers, told me. “And then we went through it again, of course. We replicated it. It's the real thing.”
Other crazy findings from the research:
Two months after the conversation with the chatbot, the researchers followed up with the participants, and their belief in the conspiracy theory hadn’t come back. They truly did change their minds.
When researchers asked the participants about other conspiracy theories – not the one the AI had specifically debunked – they believed those less, too. There was a transfer effect, which Gordon says is “a pretty unprecedented result in psychology.”
Participants were more likely to try to convince other people out of conspiracy theories on social media after the chatbot conversation.
Gordon’s team ran the experiments again and told participants that they were talking to a partisan bot – an AI model trained on Republican or Democratic data. That had NO impact. Being told the bot was biased DID NOT weaken the effects of the facts and evidence.
When researchers made it appear like participants were talking with a live expert, that also had no effect. Whether people thought they were talking to an AI model or a human, they still changed their mind the same amount.
Why was this so effective? As a journalist who has tried many times to argue sources out of conspiracy theories, I desperately wanted to know. And the answer surprised me even more than the findings themselves.
The answer is: FACTS MATTER.
Facts are what drive people's beliefs. More than the presentation or framing of the facts. More than who the messenger is and whether someone trusts them. These researchers controlled for those variables, and it still came down to facts.
Figured I'd offer that heartening finding for today. I'll save the findings about "ostensible facts" (AKA disinformation and lies) and how persuasive those also are in a chatbot for another day...
Check out my interview with one of the researchers behind this study here: open.spotify.com/episode/2Qn90c…
Or here: podcasts.apple.com/us/podcast/que…
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.