3/n The paper takes 10 countries' worth of data, and compares their COVID-19 case numbers against the restrictions they had in place in early 2020, comparing those with less-restrictive non-pharmaceutical interventions (lrNPIs) with more-restrictive NPIs (mrNPIs)
4/n The basic findings are that, compared to South Korea and Sweden, the marginal impact of implementing mrNPIs was not possible to discern in this study
In other words, "lockdowns" don't have a significant marginal benefit over a suite of other measures
5/n This has actually been shown before, using much larger datasets and more rigorous (but still imperfect) analyses, so it's a bit odd that this particular paper has been seized on so much science.sciencemag.org/content/early/…
6/n I should also say I have something of an intellectual conflict of interest here, because the authors conclude something that I've been saying since March - the interventions pursued may be less important than the way that they are implemented
7/n All that being said, what's the science here
Well, it's a bit...lacklustre really
8/n The sample size is minuscule. 8 mrNPI and 2 lnNPI countries is far too few to make any realistic conclusions about much, especially given how wildly different these countries are
9/n The authors claim that they used these countries because those were the only ones where they could get data for each administrative region, but if that's the case then they really can't have looked very hard
10/n Off the top of my head, I can name at least half a dozen other countries for which case/policy data is available by day in every administrative region. Switzerland, Australia, New Zealand, China, Brazil, etc
11/n Indeed, going by the references that the authors cite, this issue may have arisen because they used Statista as their primary source for case data, which is...not ideal
12/n The definition of restrictions is also a bit weird. I mean, South Korea didn't forcibly close businesses, but they do have national legislation allowing practices that many countries would find very restrictive (as the references the authors cite show)
13/n There's also nothing in this paper about lags for any policy for implementation, how policies were associated with dates etc
That's a huge issue!
14/n We know that the lag between policy introduction, implementation, and outcome is not immediate, and this is likely to vary by country, so simply comparing them day-by-day as this paper appears to doesn't really give us any indication of their impact
15/n Furthermore, the lrNPIs themselves are really poorly elucidated. This is FAR from a fair summation of the complex and detailed work South Korea put in to controlling COVID-19!
16/n I mean, reading the paper you might get the impression that all South Korea did was some optional social distancing, emergency declaration, and case quarantine, rather than a coordinated and multi-step approach including HUGE healthcare investment
17/n There's also not much effort to disentangle the complexities of the marginal benefit of each intervention, unlike previous research. It's likely, for example, that closing schools in Sweden (that did little else) had a huge impact...
18/n ...but that this was not as effective as in Italy, which had many interventions
19/n The authors also use some fairly inappropriate causal language throughout. These are the potential benefits ASSOCIATED WITH the announcement of policies in each place, we certainly can't infer a causal impact here
20/n In other words, there are innumerable confounding factors that may have made the interventions more/less effective, like the age structure of the population, how socially distanced they were pre-pandemic etc...
21/n At best, this study provides us with some evidence that mrNPIs are not associated with a large marginal benefit in terms of case numbers over lnNPIs, when comparing a tiny group of dissimilar nations
22/n More realistically, I think we can probably say that the paper tells us little useful except that analysing the impact of NPIs generally is extremely hard
23/n This is a bit of a shame, because I actually agree wholeheartedly with the authors that there is a cost to restrictive NPIs, and the marginal benefit of (say) stay-at-home orders is likely to be quite small in many circumstances
24/n That being said, this paper just doesn't tell us anything useful about these mrNPIs beyond some more very vague evidence that they may not be as beneficial on top of other interventions (maybe)
25/n Ultimately, the authors may have failed to find a benefit of business closures or stay-at-home orders, but the methodology used just doesn't give us enough information to say much, if anything, conclusively
26/n Some more issues with the study, which gets worse and worse the more you look at it!
The entire field of epidemiology is about balancing cost and benefit, risk and reward. There is no choice without consequences, even the seemingly trivial ones
Most Master of Public Health courses (MPH) have a health economics unit for precisely this reason. Enacting a policy in one place invariably (at best) takes away resources that you would otherwise use somewhere else
This is a big part of the reason I spend so much time trying to convey nuance. There is no decision we can make for public health that is purely good
It's likely that the marginal benefit - the additional improvement on top of other things - of very restrictive COVID-19 interventions like stay-at-home orders may be quite small
However, this is probably equally true of the COST of these interventions
It's something that I've seen completely ignored by most anti-restriction campaigners, but I think it's an important point that we should consider
Yes, if you've already limited how much people can go out to restaurants etc then closing them entirely might not reduce transmission all that much
But it also won't have the same negative impact either!
One thing that's quite funny to note about the awful @JAMANetworkOpen study that has recently been incompletely corrected is that it implies that school holidays are killing people
See, the authors assume that, in the US, every additional year of schooling reduces your risk of death by ~46% across the entire lifespan, and that any/all days missed in terms of school are precisely equivalent to missing schooling
This is not some vague sidepoint, but a central assumption underlying the entire model. Every day missed from school is precisely equivalent to missing lifetime schooling by a fixed amount per child
I do find it quite remarkable that people who have been making testable predictions that have completely failed to come through every day for MONTHS are still being given so much air
For example, a testable prediction made by Sunetra Gupta, Anders Tegnell, and others was that areas most impacted by COVID-19 in March/April would be substantially protected from any resurgence. This has proven largely wrong
This was, in part, based on the prediction by Gupta that the UK (and others) had already reached "herd immunity", or were close to it
Yes, this applies to COVID-19 as well. Stop blaming people for being sick
"But they didn't wear a mask" lots of people who DID wear a mask got COVID-19, it's not perfect protection, you can't apply morality to something that is largely out of your control