"Lockdowns only reduced mortality by 0.2%" claims (unreviewed) meta study, and it is getting much press in @FoxNews and the like.
A #metastudy aggregate the entire body of evidence. This one is based on 18,590 studies.
So, 0.2% must be a credible estimate... Well. No so fast. 🧵
A #metastudy aggregate the entire body of evidence. This one is based on 18,590 studies.
So, 0.2% must be a credible estimate... Well. No so fast. 🧵
https://twitter.com/CaulfieldTim/status/1489318492780171268
A meta study often begins with a large set of studies (18,590) then eliminates irrelevant studies, duplicates, etc.
This meta study ends up with 34 papers.
Thus 0.2% is based on 34 studies, right? Well, actually not.
This meta study ends up with 34 papers.
Thus 0.2% is based on 34 studies, right? Well, actually not.
Somewhat oddly the number is not 34, but 24.
There are 34 studies listed in Table 2, but 10 of them are listed as excluded. Apparently based on some criterion that is missing in Figure 2.
OK. But 0.2% based on 24 studies still gives a precise estimate, right? Sadly no.
There are 34 studies listed in Table 2, but 10 of them are listed as excluded. Apparently based on some criterion that is missing in Figure 2.
OK. But 0.2% based on 24 studies still gives a precise estimate, right? Sadly no.
The 0.2% is not based on 24 studies, just 7 studies listed in Table 3.
But wait a minute.
The estimates ranges from -35.3% to +0.1% (highlighted in yellow).
How do the authors end up with -0.2%?
But wait a minute.
The estimates ranges from -35.3% to +0.1% (highlighted in yellow).
How do the authors end up with -0.2%?
Weighting, is the answer.
The only study to find lockdowns ⬆️mortality is given weight 91.8% = 7390/8030, and then you get -0.2% to be the estimate.
To summarize: -0.2% META-STUDY ESTIMATE is based on 91.8% ONE STUDY and 8.2% ALL OTHER STUDIES.
The only study to find lockdowns ⬆️mortality is given weight 91.8% = 7390/8030, and then you get -0.2% to be the estimate.
To summarize: -0.2% META-STUDY ESTIMATE is based on 91.8% ONE STUDY and 8.2% ALL OTHER STUDIES.
Where was the study given 91.8% weight published?
In MDPI Sustainability. MDPI is a controversial outlet, was classified as predatory journals. Median time from submission to publication is 39 days and
MDPI Sustainability published 7,414 article in 2019.
Impressive!
In MDPI Sustainability. MDPI is a controversial outlet, was classified as predatory journals. Median time from submission to publication is 39 days and
MDPI Sustainability published 7,414 article in 2019.
Impressive!
What happened to the 17 = 24 - 7 other papers in the "meta analysis"?
Well they are other Tables, such as this one, with some oddities, I might follow up on later.
Well they are other Tables, such as this one, with some oddities, I might follow up on later.
Turns out @videnskabdk wrote about this “meta study” before me, in which Mathias Heltberg (PostDoc at @uni_copenhagen) makes many of the same comments (in 🇩🇰).
He also makes an observation about the MDPI study that I overlooked. It concluded the opposite!
videnskab.dk/krop-sundhed/f…
He also makes an observation about the MDPI study that I overlooked. It concluded the opposite!
videnskab.dk/krop-sundhed/f…
Central point in the paper, which was weighted 91.8%, is the importance of NON-LINEARITY. The authors conclude: "interventions at higher levels of severity reduce deaths".
The "meta study" ignored this and just extracted a linear term. That is "creative".
The "meta study" ignored this and just extracted a linear term. That is "creative".
I found a bit more information about the journal, MDPI Sustainability, which published the 91.8%-weighted article.
Authored by @maoviedogarcia
academic.oup.com/rev/article/30…
Authored by @maoviedogarcia
academic.oup.com/rev/article/30…
Another great thread is this one by @whippletom
https://twitter.com/whippletom/status/1489197128916389894
• • •
Missing some Tweet in this thread? You can try to
force a refresh
