Thread by @GidMK

1/n Some more movement on this study from earlier in the year by Drs Ioannidis, Bhattacharya, Oh, and Bendavid

Along with @lonnibesancon, @FLAHAULT, and others, we've published a series of responses and ongoing critiques of the piece

https://twitter.com/GidMK/status/1349164532627693570

2/n The newest response is here, and you can have a look at the previous discussion as well:

onlinelibrary.wiley.com/doi/10.1111/ec…

3/n For reference, if you've forgotten, the original article basically argued that "more" restrictive non-pharmaceutical interventions (mrNPIs) such as lockdowns didn't work to prevent COVID-19 cases

It is MASSIVELY popular, with an Altmetric of 19k and dozens of citations

4/n Our basic arguments are pretty simple:

1. Sample size - this study uses a very small sample. That's not really debated by the authors, and it's a limitation

5/n (Weirdly, they have also contradicted their original paper in the replies to the piece - where before they said that their paper had data for every country available, they're now saying they applied a strange exclusion)

6/n 2. The classification of countries as "high" or "low" restriction is arbitrary. In their reply, the authors literally just contradicted us, so we've now got quite a bit of text demonstrating why this is an issue

7/n This is pretty simple - if you don't explain why (say) South Korea is a "low" restriction country, how can we compare it to "high" restriction ones? The categorization is fundamental to the entire paper

8/n 3. Issues with the model. In particular, the policy variable

9/n 4. Issues with the use of time lags, and the time period studied

In particular, the authors say that they analyzed the data up until the "elimination of rapid growth in the first wave"

10/n This is a problem because it means that the analysis is entirely limited to the 'upwards' part of the epidemic curve

If mrNPIs have an effect that is mostly seen on the decline, this would by definition be missed

11/n 5. While we also had some quibbles about language, the final major argument is to do with the model again

12/n Ultimately, we still have the same overall gripe. It is pretty much impossible from the evidence presented in the original research to conclude that mrNPIs work or do not

13/n Indeed, as we point out, unless the authors spend some time delineating exactly what a "more" or "less" restrictive response to COVID-19 actually is, there isn't much you can take home from the analysis anyway

14/n My personal position remains mostly the same on these interventions, that more restrictive ones probably weren't that beneficial in the early days of the pandemic, but that it's really, really hard to know either way