Thread by @whippletom

I'm going through this paper by Johns Hopkins economists, that assesses the efficacy of lockdown in the US and Europe - and concludes it was essentially useless.

I'd love thoughts on something I've found, which may well be my misinterpretation 1/x sites.krieger.jhu.edu/iae/files/2022…

The study is a meta-analysis, combining other previous between-country comparisons. To produce the combined estimate, they take 18,000 studies, exclude all but a handful and then pool their findings.

Looking at the weighting, it actually seems to be based almost entirely on one study. So rather than being a meta-analysis it is really a recapitulation of that study, by Chisadza et al

The other odd thing is that study itself comes to a very different conclusion about the efficacy of lockdown.

mdpi.com/2071-1050/13/6…

The JH study does mention they have a difference of opinion. But, it does seem odd - if indeed this study is the sole basis for their study - that they differ so widely.

It also struck me as strange to exclude the other data from other regions that is in the chisadza paper. If your meta analysis includes other papers, then obviously you have to take the relevant bit of each. But if the meta analysis is just a reanalysis, why lose info?

(unless I've misunderstood in particular what the weighting means which - again - I may well have. This thread is me thinking aloud, hoping for answers!)

I spoke to the author of the paper on whose research this entire meta-analysis was based. She said: "They already had their hypothesis. They think that lockdown had no effect on mortality, and that’s what they set out to show in their paper."

A couple of studies from high impact journals that somehow didn’t make the cut, when the authors decided of 18,000 papers effectively just one met their exacting standards:

From Nature Human Behaviour

"The social distancing and movement-restriction measures discussed above can therefore be seen as the ‘nuclear option’ of NPIs: highly effective but causing substantial collateral damage"

nature.com/articles/s4156…

From Science

Limiting gatherings to fewer than 10 people, closing high-exposure businesses, and closing schools and universities were each more effective than stay-at-home orders, which were of modest effect in slowing transmission.

science.org/doi/10.1126/sc…