In our new pre-print, we find significant evidence of user radicalization on #YouTube. Tracking >79M comments across >300k videos, we show that those who comment on the so-called Intellectual Dark Web/Alt-lite consistently migrate to Alt-right channels. arxiv.org/abs/1908.08313
Really grateful towards my co-authors @raphaottoni , @cervisiarius, @virgilioalmeida , @wagnermeirajr . Also, thanks @jhblackb (a.k.a. Mr. Jimmy Black) for giving me tons of feedback. If you want a tl;dr on the paper, stick with me for a few more tweets below! 👇👇👇
In late 2017, I got surprised by the presence of fringe ideologies (e.g. White Supremacy) on YouTube. In 2018 I got redirected to one of such channels from an "Anti-SJW"-style video. Later that year, when I read @zeynep and @beccalew pieces on the alleged phenomena of...
...user radicalization on YouTube it all just clicked in my head. I still thought that the evidence indicating this phenomenon was happening was very anecdotal. Spent months devising a careful data collection method with @raphaottoni. By the end of this period...
...we collected and annotated a YouTube dataset of unprecedented size. 360 channels, more than 79 million comments, more than 2 million recommendations. We broadly categorized channels into Alt-right, Alt-lite, and Intellectual Dark Web...
...These communities were broadly defined by non-profits, by the media, and by the own content creators. We also collected control channels from large media outlets. Our findings blew my mind a little bit:
1) We looked at the growth of the I.D.W., the Alt-lite and the Alt-right throughout the last decade in terms of videos, likes, and views, finding a steep rise in activity and engagement in the communities of interest when compared with the control channels;
2) We inspect the intersection of commenting users within the communities, finding they increasingly share the same user base. For example, we find that more than half of users who commented on Alt-right channels in 2018 also comment on Alt-lite and on I.D.W. channels;
3) We also find that the intersection is not only growing due to new users but that there is significant user migration among the communities being studied. Users that initially comment only on content from the I.D.W. or the Alt-lite throughout the years consistently start to...
> on Alt-right content. These users are a significant fraction of the Alt-right commenting user base. Interestingly, although control channels share, on a yearly basis, a significant number of users with Alt-right channels, we cannot observe significant user migration from them.
4) Lastly, we take a look at the impact of YouTube's recommendation algorithms, running simulations on recommendation graphs. Our analyses show that, particularly through the channel recommender system,...
> Alt-lite channels are easily discovered from I.D.W. channels, and that Alt-right channels may be reached from the two other communities (!).
I believe our research has implications for content creators and for YouTube:
a) I believe that popular content creators like those in the Intellectual Dark Web, should be aware that this process is happening I genuinely believe this is not what they want, and they are in a good position to try to weaken fringe ideologies like white supremacy.
b) YouTube has to be held accountable and take action. I believe our work presents significant evidence that a large scale radicalization happened on the website. This process has led to horrific acts of terrorism, and should be addressed accordingly.
🚨LLMs outdebate humans when given access to opponents' personal Information!🚨
In a pre-reg study (N=820), participants who debated ChatGPT had 81.7% (p<0.01) higher odds of agreeing with their opponents after the debate (compared to a human baseline)
We ran a synchronous experiment (@empirica_ly!) where participants were matched with another participant or an LLM to debate,
Some participants/LLMs were provided background information about their opponents, and we measured agreement with the debate statement pre vs post debate
Using ordinal regression, we compare how participants change their opinion about the proposition of interest pre-vs-post debate, depending on whether they debated a human or an AI and whether their opponent had personal information, eg, their gender, age, and political position.
One of our key sources of human data is no longer fully “human"!
We estimate that 33-46% of crowd workers on MTurk used large language models (LLMs) in a text production task - which may increase as ChatGPT and the like become more popular and powerful.
We re-run a human intelligence task from Horta Ribeiro et al. 2019 (yours truly): to summarize abstracts from the New England Journal of Medicine. This is a laborious task that is within reach of current LLMs, like ChatGPT, Claude, Bard, etc.
Using answers from the original study and data synthetically generated with ChatGPT, we train a bespoke synthetic-real classifier that works remarkably well. We use this classifier to estimate the prevalence of synthetic answers in the task we reran.