Thread by @benwansell

🚨The problems with democracy coding and bias 🚨 Political scientists among you will know about the Polity IV score. This has been until recently the preferred measure of democracy for many scholars. So why, you may ask, does it not like democracy in US or UK? 1/n

Until quite recently - I'll let you guess the year - Britain and the US were coded as +10 on the Polity score. That's the max in a -10 to +10 scale. Same as Sweden, Germany, Spain, Portugal, Poland, Hungary... 2/n

Polity is made up of 3 components: constraints on the executive, recruitment of the executive, and political competition. The former is checks & balances, middle is how leaders are chosen, and latter is about elections and party competition. Final index comprised of all 3. 3/n

Now how they are combined is hilarious - all kinds of odd configurations that would give you serious pause about ever using Polity. But basically if you have +10 on the Polity index it means you are maxing out on the three components. So what's happened recently? 4/n

In the last couple of years the Polity peeps have recoded the US and UK as +8 on the Polity score. They have become LESS democratic according to one of our most-used measures. And what's the year of infamy? 2016. Gosh, what could have happened then? 5/n

For reasons best known to the coders, the temerity of the British electorate to vote to leave the EU has knocked the UK's political competitiveness score down from 10 to 7 because the UK has become 'factional'. Because the main index is weird that pushes it down from 10 to 8. 6/n

The US was similarly punished for electing Trump (by electoral college not by popular vote, though I don't think that's the reasoning). Exact same pattern of downgrading. Now this means the US IS LESS DEMOCRATIC THAN IN 1845!!!. 7/n

The US was similarly punished by Polity during the Nixon administration. But not during FDR and court-curbing or indeed any period between 1871 and 1968. The UK for its part has not been this low since 1922. 8/n

Call me crazy but the entire era of Jim Crow in the US doesn't seem to me to be more democratic than 2017-19. The era in which British business people and Oxford grads were given 2 votes (or in which women under 30 couldn't vote) doesn't seem more democratic than last week. 9/n

Well maybe this is just the Polity score coming to terms with populism. Maybe democracy has really got noticeably worse (than 1845/1922...). So why are Poland and Hungary still coded as +10 in the Polity index?!? And is Mexico really just as democratic today as UK and US? 10/n

This type of coding is just plain off. And ultimately has to reflect coders downgrading the US and the UK because people voted in a way they didn't like but not doing the same for Poland, Hungary, Italy, etc. And it undermines trust in these indices in the first place 11/n

I understand why some political scientists want to look at 'red lines' in the US and elsewhere. But these kinds of decisions undermine scholars of democracy when we want to talk about objective indicators and make us look politically biased. 12/n

I have used the Polity score many times in my work. On education policy (amazon.co.uk/Ballot-Blackbo…) and indeed on the (positive!) effects of inequality on democratization amazon.co.uk/gp/product/B00… . I feel much less confident in doing so if this is what I have to work with. 13/n

So fellow scholars of democracy, I suggest we take a deep breath about announcing 'threats to democracy' everywhere, particularly if elections we don't like happen. Losing elections and losing democracy are not the same thing. n/n

Postscript - at least Polity has open data and you can look at their coding and justification at systemicpeace.org/inscrdata.html. Britain downrated because referendum "served to polarize politics in the UK (change POLCOMP to 7 "factional"). Uh huh...

PPS: Thread maybe of interest to @Samuels_DavidJ @SlaterPolitics @TomPepinsky @AnnaGBusse @StaffanILindber @simonjhix @sarahobolt @MilanSvolik @KTertytchnaya