Thread by @BaldingsWorld

This is the original WaPo trash piece about whether Biden gets more negative coverage than Trump. Let me explain why it is trash and warn you now this has near zero to do with anything partisan so just warning you now 1/n
washingtonpost.com/opinions/2021/…

Sentiment analysis is a run of the mill tool, and there are different flavors out there, that seems to classify a language dataset as more positive or negative. The WaPo article doesn't specify which one they used but I'll assume it's one of the standard ones and no funny biz 2/n

It sounds complicated but what it does, (oversimplifying here), is count up positive and negative words and then calculate whether the article was more positive or negative. Different flavors can tweak the calculation in different ways that are pretty accepted but pretty 3/n

Straightforward really. However, as anyone who has actually worked with this knows sentiment analysis is a very crude, noisy, blunt tool that can be very misleading for so so so many reasons. Again this isn't partisan so let me explain some if the ways this CAN be misleading 4/n

Just to throw out SOME ways: first, positive and negative words or phrases have an enormous range of grades. For instance, you SUCK and you need to improve are both negative but carry very different meanings. Most sentiment analysis struggles to actually capture these grades 5/n

Second, why is negative language being used? As some public institutions (say the IMF) like to argue, we get blamed because we are the ones who show up to tragedies. Sentiment analysis does nothing to disentangle being in or near the accident and causing the accident 6/n

For instance, all Presidents get lots of negative press regardless of how good they are. They are seen as THE responsible party. The buck stops there. If you will notice in the article, Biden numbers drop precipitously when Afghanistan happens. People can have differences 7/n

About causation and responsibility but any president overseeing those events would have received significantly negative coverage. There are timing and causation issues. Third, the last primary issues I'll hit here are subjective/objective in sentiment language analysis. 8/n

WaPo writes "My colleagues in the media are serving as accessories to the murder of democracy." This would be a rhetorical subjective statement. There is no actual murder of democracy. The sky is blue would be an objective statement. One important aspect of doing analysis 9/n

Like WaPo so very poorly tried to do is to capture the multidimensionality of language and intent. For instance, the first example cited is "Let the Democratic freakout begin" which is negative expressing worry coded as negative but isn't critical of Biden. To accurately 10/n

Capture whether Biden is being treated better or worse than Trump or some semi objective baseline, there is a lot lot lot more work that needs to be done than something an experienced coder could band out in an afternoon. I make no determination on the question on Biden treatment