Health Nerd Profile picture
13 Jan, 26 tweets, 7 min read
A new paper has been published by John Ioannidis and Jay "Great Barrington Declaration" Bhattacharya on "lockdowns" as a COVID-19 preventative measure

Let's do some twitter peer-review! 1/n
2/n The paper is here, and it's an interesting read:

onlinelibrary.wiley.com/doi/epdf/10.11…
3/n The paper takes 10 countries' worth of data, and compares their COVID-19 case numbers against the restrictions they had in place in early 2020, comparing those with less-restrictive non-pharmaceutical interventions (lrNPIs) with more-restrictive NPIs (mrNPIs) Image
4/n The basic findings are that, compared to South Korea and Sweden, the marginal impact of implementing mrNPIs was not possible to discern in this study

In other words, "lockdowns" don't have a significant marginal benefit over a suite of other measures Image
5/n This has actually been shown before, using much larger datasets and more rigorous (but still imperfect) analyses, so it's a bit odd that this particular paper has been seized on so much science.sciencemag.org/content/early/… Image
6/n I should also say I have something of an intellectual conflict of interest here, because the authors conclude something that I've been saying since March - the interventions pursued may be less important than the way that they are implemented Image
7/n All that being said, what's the science here

Well, it's a bit...lacklustre really
8/n The sample size is minuscule. 8 mrNPI and 2 lnNPI countries is far too few to make any realistic conclusions about much, especially given how wildly different these countries are Image
9/n The authors claim that they used these countries because those were the only ones where they could get data for each administrative region, but if that's the case then they really can't have looked very hard Image
10/n Off the top of my head, I can name at least half a dozen other countries for which case/policy data is available by day in every administrative region. Switzerland, Australia, New Zealand, China, Brazil, etc
11/n Indeed, going by the references that the authors cite, this issue may have arisen because they used Statista as their primary source for case data, which is...not ideal Image
12/n The definition of restrictions is also a bit weird. I mean, South Korea didn't forcibly close businesses, but they do have national legislation allowing practices that many countries would find very restrictive (as the references the authors cite show) Image
13/n There's also nothing in this paper about lags for any policy for implementation, how policies were associated with dates etc

That's a huge issue!
14/n We know that the lag between policy introduction, implementation, and outcome is not immediate, and this is likely to vary by country, so simply comparing them day-by-day as this paper appears to doesn't really give us any indication of their impact
15/n Furthermore, the lrNPIs themselves are really poorly elucidated. This is FAR from a fair summation of the complex and detailed work South Korea put in to controlling COVID-19! Image
16/n I mean, reading the paper you might get the impression that all South Korea did was some optional social distancing, emergency declaration, and case quarantine, rather than a coordinated and multi-step approach including HUGE healthcare investment
17/n There's also not much effort to disentangle the complexities of the marginal benefit of each intervention, unlike previous research. It's likely, for example, that closing schools in Sweden (that did little else) had a huge impact...
18/n ...but that this was not as effective as in Italy, which had many interventions
19/n The authors also use some fairly inappropriate causal language throughout. These are the potential benefits ASSOCIATED WITH the announcement of policies in each place, we certainly can't infer a causal impact here Image
20/n In other words, there are innumerable confounding factors that may have made the interventions more/less effective, like the age structure of the population, how socially distanced they were pre-pandemic etc...
21/n At best, this study provides us with some evidence that mrNPIs are not associated with a large marginal benefit in terms of case numbers over lnNPIs, when comparing a tiny group of dissimilar nations
22/n More realistically, I think we can probably say that the paper tells us little useful except that analysing the impact of NPIs generally is extremely hard
23/n This is a bit of a shame, because I actually agree wholeheartedly with the authors that there is a cost to restrictive NPIs, and the marginal benefit of (say) stay-at-home orders is likely to be quite small in many circumstances
24/n That being said, this paper just doesn't tell us anything useful about these mrNPIs beyond some more very vague evidence that they may not be as beneficial on top of other interventions (maybe)
25/n Ultimately, the authors may have failed to find a benefit of business closures or stay-at-home orders, but the methodology used just doesn't give us enough information to say much, if anything, conclusively Image
26/n Some more issues with the study, which gets worse and worse the more you look at it!

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Health Nerd

Health Nerd Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @GidMK

25 Jan
The entire field of epidemiology is about balancing cost and benefit, risk and reward. There is no choice without consequences, even the seemingly trivial ones
Most Master of Public Health courses (MPH) have a health economics unit for precisely this reason. Enacting a policy in one place invariably (at best) takes away resources that you would otherwise use somewhere else
This is a big part of the reason I spend so much time trying to convey nuance. There is no decision we can make for public health that is purely good

There are no silver bullets
Read 5 tweets
14 Jan
It's likely that the marginal benefit - the additional improvement on top of other things - of very restrictive COVID-19 interventions like stay-at-home orders may be quite small

However, this is probably equally true of the COST of these interventions
It's something that I've seen completely ignored by most anti-restriction campaigners, but I think it's an important point that we should consider
Yes, if you've already limited how much people can go out to restaurants etc then closing them entirely might not reduce transmission all that much

But it also won't have the same negative impact either!
Read 4 tweets
11 Jan
One thing that's quite funny to note about the awful @JAMANetworkOpen study that has recently been incompletely corrected is that it implies that school holidays are killing people
See, the authors assume that, in the US, every additional year of schooling reduces your risk of death by ~46% across the entire lifespan, and that any/all days missed in terms of school are precisely equivalent to missing schooling
This is not some vague sidepoint, but a central assumption underlying the entire model. Every day missed from school is precisely equivalent to missing lifetime schooling by a fixed amount per child
Read 8 tweets
11 Jan
I do find it quite remarkable that people who have been making testable predictions that have completely failed to come through every day for MONTHS are still being given so much air
For example, a testable prediction made by Sunetra Gupta, Anders Tegnell, and others was that areas most impacted by COVID-19 in March/April would be substantially protected from any resurgence. This has proven largely wrong
This was, in part, based on the prediction by Gupta that the UK (and others) had already reached "herd immunity", or were close to it

Also wrong
Read 4 tweets
11 Jan
Some movement to announce here: JAMA Open have now corrected this paper 2 months after it was published

Unfortunately, it has gone from an error-filled useless analysis to a slightly less error-filled useless analysis

Some more peer-review on twitter 1/n
2/n The updated paper is here jamanetwork.com/journals/jaman…

And you can read @ikashnitsky and my original commentary on the paper here

osf.io/9yqxw/
2.5/n Important to note that this is a very influential paper. It has been in >100 news stories, and has been cited by the EU and WHO (!)

Worrying that until recently it was openly wrong
Read 33 tweets
10 Jan
GETTING SICK IS NOT A MORAL FAILING
Yes, this applies to COVID-19 as well. Stop blaming people for being sick
"But they didn't wear a mask" lots of people who DID wear a mask got COVID-19, it's not perfect protection, you can't apply morality to something that is largely out of your control
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!