Thread by @J_Lovering

1/ I know there are serious problems in the world, but bad data journalism can make them worse. I didn’t want to wade into #BernieBro discourse, but I can’t let bad analysis slide.
Specifically, this article: salon.com/2020/03/09/the…
Thread:

2/ I was surprised how many people I respect shared this article, considering the results of the analysis DO NOT support the headline. Perhaps people did not read the article too carefully, but I also worry when people share an article just for the headline. Read for yourself...

3/ The actual analysis in the article was done by Harvard grad student @CompSocialSci, & it's fine analysis. Here’s what they did: Evaluated the sentiment (positive to negative) of all tweets since 2015 of a random selection of followers of each of top 9 dem primary candidates.

4/ Result: Share of positive vs. negative tweets all about the same across candidate’s followers. So...Bernie Bros are a myth! No. Let me explain.
-One big assumption in such an analysis is that the groups you are comparing are independent, but in reality...

5/ ... people follow more than one candidate on twitter, you may follow all of them! You could be sampling the same people.
-Now you could do a simple test to see how much overlap there is across follower groups. @CompSocialSci should do that. But...

6/ More importantly, IMHO, followers ≠ supporters. How many of you follow the president, for example.
How to correct for this ?
-You can start with candidate followers, but then narrow down to supporters, either by keywords in profile bios or links to candidate in bio

7/ Or you could impose some threshold number of tweets referencing the candidate.
-Also, this analysis doesn’t separate out bots. (& I’m super curious to see the sentiment analysis of bots, but nvm). You can use a tool like @BotSentinel to filter out bots first.

8/ The author makes a repeated point that @BernieSanders just has a lot more followers, so that may be skewing people’s perception of his followers. But he also has a lot more bot followers, about twice the share of bots:

9/ @CompSocialSci does make a point that they should narrow down the tweets analyzed to just relevant one, maybe those referencing another candidate, or you could narrow down timeframe to around primary, etc. People could be tweeting negative things about poptarts for all we know

10/ Now, if you ensured independence of the groups you analysis, and then also make sure the tweets you analyze are politically relevant, and still found no difference between diff. candidates followers, THEN you could write that bold headline and I would be more convinced.

11/ BUT, a lot of the harassment that women in particular attribute to Bernie Bros is in private messages, you've seen the screenshots, so it's harder to include in this kind of analysis. Headline should be: "Data shows no difference in followers of each candidate"