Thread by @GidMK

Wow. Yet another ecological paper on COVID-19 that appears to have some astonishing flaws

Let's do a bit of peer-review on twitter 1/n

https://twitter.com/ClarkeMicah/status/1331959052016947201

2/n Paper is here. Basically, the authors took total COVID-19 mortality in a range of countries by August 31st and correlated them with country-level metrics such as average BMI frontiersin.org/articles/10.33…

3/n The authors found that country-level metrics compiled by the University of Oxford on the stringency of lockdowns did not correlate well with the number of deaths that a country experienced from COVID-19, but other things like BMI and latitude did

4/n But even a very shallow skim of the methodology brings up extremely serious concerns

For example, where did they get their data?

5/n Well, the obesity rates were taken from the CIA Factbook

This is manifestly inappropriate. The factbook was last updated in 2016 for these figures and mostly references reports that are 6+ YEARS old

6/n I'm not sure what meaning we could take from correlating COVID-19 deaths in August with obesity rates from 2012, but it's certainly not a lot

7/n The physical inactivity data was taken from the WHO's global data repository, which was updated in 2018. This is better, but it's still aggregating data from vastly different surveys done in different ways in different countries across the world

8/n The government stringency index was taken from Oxford data, which you can see here. You can see some problems with this almost immediately as well

9/n For example, in August (when this study compared countries), Australia was rated as 75/100 on the stringency index. This is because one state of the country (Victoria) had extremely strict restrictions in place

10/n But for most of Australia, life was largely back to normal (except for travel) by August!

Similarly, India is given a stringency index of 81(!) even though the response there varies quite a bit by states as well

11/n Which brings us to a more central issue with this entire analysis - the ecological fallacy

I've written about this before gidmk.medium.com/why-you-might-…

12/n Simply comparing COVID-19 death rates at the national level with some population-level indicators is a pretty pointless thing to do no matter how you do it

13/n What possible meaning can you garner from the fact that the latitude of the barycenter of some countries correlates with their COVID-19 death toll as of August 31st?

14/n Even worse, the authors made no attempt to control for the age distribution of those infected in the population, which as we know very well by now is the biggest defining factor in the death rate from COVID-19

15/n Also, the authors note that reverse causality may be an issue when interpreting the correlation between COVID-19 deaths and economic intervention, but don't make this point for the stringency index even though it's probably exactly as problematic

16/n Also, the fact that the authors call this "the least uncertain" data is pretty wild, because a) it isn't and b) the least uncertain sources are TERRIBLE for many countries so WHY USE THEM AT ALL????

17/n Ultimately, I cannot see how you can take home anything from this research aside from the fact that some country-level measures recorded years ago correlate well with COVID-19 deaths and some don't

18/n I feel like @TylerVigen should create a new version of the Spurious Correlations website with COVID-19 deaths so that people can see just how pointless all of these ecological studies really are