Thread by @mark_ledwich

1/ There are two main criticisms about the study that I should have anticipated better.

a) anon recommendations could be very different from personalized ones
b) the NYT and others were reporting on 2018 radicalization, yet we only analyzed late 2019 recommendations

2/ Anonymous recs (even when averaged out over all users) could have a different influence compared to personalized ones. This is a legit limitation, but one that applies to all of the studies on recs so far.

3/ There are practical reasons that make this extremely difficult - you would need a chrome extension (or equivalent) that captures real recommendations in click-through stats from a representative set of users. I don't plan on doing that.

4/ I plan on begging for stats from youtube creators to export data ... from their "YouTube analytics traffic sources -> suggested video" reports to comparing it with our data. In the meantime, you should accept this as the best quality data you have to go on.

5/ As for b) here is monthly data since Dec 2019. It's not totally clean - there were channels added and some minor changes in processes throughout that period - it slightly noisy but representative of what happened over that period.

6/ Pew's study from June 2018 shows that recs, broadly lead to more popular, longer videos which is consistent with the early 2019 data we have.
pewresearch.org/internet/2018/…

7/ The Auditing Radicalization Pathways study (despite the way it was reported) also found recommendations leading away from alt-right and alt-light. arxiv.org/pdf/1908.08313…

8/ Recommendation-rabbit-hole proponents have been ignoring evidence and searching for compelling anecdotes since 2018.

I wish you all the best in dealing with your dissonance. I hope it's not too painful.

END