🚨 New today in @ScienceMagazine !!🚨

We’re publishing the results of the largest AI persuasion experiments to date: 76k participants, 19  LLMs, 707 political issues

We examine “levers” of AI persuasion: model scale, post-training, prompting, personalization, & more…

🧵: Findings (pp = percentage points):

1️⃣Scale increases persuasion, +1.6pp per OOM
2️⃣Post-training more so, as much as +3.5pp
3️⃣Personalization less so, <1pp
4️⃣Information density drives persuasion gains
5️⃣Increasing persuasion decreased factual accuracy 🤯
6️⃣Convo > static, +40%

Today (w/ @UniofOxford @Stanford @MIT @LSEnews) we’re sharing the results of the largest AI persuasion experiments to date: 76k participants, 19  LLMs, 707 political issues.

We examine “levers” of AI persuasion: model scale, post-training, prompting, personalization, & more
🧵 RESULTS (pp = percentage points):

1️⃣Scale increases persuasion, +1.6pp per OOM

2️⃣Post-training more so, +3.5pp 

3️⃣Personalization less so, <1pp

4️⃣Information density drives persuasion gains

5️⃣Increasing persuasion decreased factual accuracy 🤯

6️⃣Convo > static, +40%

‼️New preprint: Scaling laws for political persuasion with LLMs‼️

In a large pre-registered experiment (n=25,982), we find evidence that scaling the size of language models yields sharply diminishing persuasive returns:



1/n arxiv.org/abs/2406.14508
We find that: 

➡️ current frontier models (GPT-4, Claude-3) are barely more persuasive than models smaller in size by an order of magnitude or more, and  

➡️ mere task completion (coherence, staying on topic) appears to account for larger models' persuasive advantage.

🚨New today in @PNASNews ‼️w/ @helenmargetts:

In a pre-registered experiment (n=8,587), we find little evidence that 1:1 personalization — aka microtargeting — enhances the persuasive influence of political messages generated by GPT-4. 



1/7 👇🏼 pnas.org/doi/10.1073/pn…
Our findings suggest:

1️⃣Personalizing static political messages with current frontier LLMs may not offer the persuasive advantage that has been widely speculated…

2️⃣but both targeted and non-targeted messages are broadly persuasive.

How we tested this:

🚨 New paper! 🚨 w/ @william__brady & @manos_tsakiris:

Today in @PNASNexus we map the moral language of 39 U.S. presidential candidates to show how they're connected / differentiated by their use of moral rhetoric.
ow.ly/8Zwu50OO5QA

A [visual] thread on findings: Main takeaways:

1) @TheDemocrats & @GOP
candidates use sharply divergent moral vocabularies

2) Candidates can separate themselves from the rhetorical norms of their party by using unique moral language, but rarely do