🚨LLMs outdebate humans when given access to opponents' personal Information!🚨

In a pre-reg study (N=820), participants who debated ChatGPT had 81.7% (p<0.01) higher odds of agreeing with their opponents after the debate (compared to a human baseline)

📜 arxiv.org/abs/2403.14380
We ran a synchronous experiment (@empirica_ly!) where participants were matched with another participant or an LLM to debate,

Some participants/LLMs were provided background information about their opponents, and we measured agreement with the debate statement pre vs post debate

Is the YouTube algorithm radicalizing us?

Our new PNAS paper says: probably not!

But why should you trust this paper rather than other papers or reports saying otherwise?

Because we came up with a way to disentangle the causal effect of the algorithm.

pnas.org/doi/10.1073/pn…
We call the new approach counterfactual bots.

It needs two ingredients:
1: real user traces (here YT histories)
2: bots—computer programs simulating user behavior

For each YT history, we instantiate various bots. One is a "control bot" that watches the same videos as the user.

One of our key sources of human data is no longer fully “human"!

We estimate that 33-46% of crowd workers on MTurk used large language models (LLMs) in a text production task - which may increase as ChatGPT and the like become more popular and powerful.

arxiv.org/abs/2306.07899 We re-run a human intelligence task from Horta Ribeiro et al. 2019 (yours truly): to summarize abstracts from the New England Journal of Medicine. This is a laborious task that is within reach of current LLMs, like ChatGPT, Claude, Bard, etc.

Just finished the "research highlights" for the papers I will present at @icwsm 2021! Thought they could also be cool to share on Twitter :D The Evolution of the Manosphere Across the Web
📜: arxiv.org/abs/2001.07600
🗄️: zenodo.org/record/4007913
With @jhblackb, barry, @emilianoucl, @gianluca_string, @slong121, stephanie and @zsavvas90

In our new pre-print, we find significant evidence of user radicalization on #YouTube. Tracking >79M comments across >300k videos, we show that those who comment on the so-called Intellectual Dark Web/Alt-lite consistently migrate to Alt-right channels. arxiv.org/abs/1908.08313 Really grateful towards my co-authors @raphaottoni , @cervisiarius, @virgilioalmeida , @wagnermeirajr . Also, thanks @jhblackb (a.k.a. Mr. Jimmy Black) for giving me tons of feedback. If you want a tl;dr on the paper, stick with me for a few more tweets below! 👇👇👇