1/
Many COVID-19 contrarians, including those behind the Great Barrington Declaration, *still* cite John Ioannidis' inaccurate estimate of SARS-CoV-2's fatality rate.
So let's go over how atrocious Ioannidis' paper is.
web.archive.org/web/2020111809…
2/
Background:
When a virus infects u, your body increases production of proteins known as antibodies, which are usually specific to that virus.
So measuring antibodies lets u estimate who was infected, and from that the infection fatality rate (IFR).
institutefordiseasemodeling.github.io/nCoV-public/an…
3/
Ioannidis uses antibody (a.k.a. seroprevalence) studies to estimate the number of people infected with the virus SARS-CoV-2. He then calculates IFR by dividing the number of COVID-19 deaths by the number of infected people.
Ioannidis does this badly:
medrxiv.org/content/10.110…
4/
It's no secret in the scientific community that Ioannidis' paper is poor. Some examples:
rapidreviewscovid19.mitpress.mit.edu/pub/p6tto8hl/r…
rapidreviewscovid19.mitpress.mit.edu/pub/6z09ucch/r…
rapidreviewscovid19.mitpress.mit.edu/pub/k2pkhufi/r…
hildabastian.net/index.php/91
web.archive.org/web/2020121700…
5/
Ioannidis' paper gives a median IFR estimate of 0.27%, which he corrects to 0.23%. That's a low outlier.
"[..] 0.79% (95% credible interval, 0.68–0.92%) [...], with a median range of 0.24–1.49%"
nature.com/articles/s4158…
web.archive.org/web/2020121700…
6/
Beware when folks cherry-pick Ioannidis' outlier of ~0.23% as if its credible.
Public Health Agency of Sweden:
"Globally, it is estimated that 0.5–1 percent of those who are infected with COVID-19 die"
web.archive.org/web/2020103000…
sciencedirect.com/science/articl…
7/
Also beware of people who try to mislead you into thinking the World Health Organization agrees with Ioanndis' low IFR estimate, or with his claims that SARS-CoV-2 has an IFR similar to that of seasonal influenza.
October 12:
who.int/publications/m…
8/
The points above have been made before. So what will this thread add?
Well, I'll go through each of the 61 studies Ioannidis cites to support his IFR estimate, + show they don't adequately support his estimate.
A numbered list of the studies:
web.archive.org/web/2020111809…
9/
Antibody studies can use a *representative* sample of the population to accurately estimate the number of infections. Otherwise, one could over-estimate the number of infections + under-estimate IFR.
Scientists know how to get representative samples.
sciencedirect.com/science/articl…
10/
Re: "Scientists know how to get representative samples."
For instance:
link.springer.com/article/10.100…
wwwnc.cdc.gov/eid/article/26…
link.springer.com/article/10.100…
cdc.gov/nceh/casper/ov…
web.archive.org/web/2020101623…
sci-hub.se/https://www.jc…
ncbi.nlm.nih.gov/pmc/articles/P…
11/
Ioannidis mostly cites studies that use non-representative sampling. That causes him to over-estimate seroprevalence + thus under-estimate IFR.
Studies with non-representative sampling may be useful for other purposes, but not for estimating population-wide IFR.
12/
For instance, blood donor studies:
- under-sample older people, a population more likely to die of COVID-19
- include people who go outside to donate blood, + are thus more likely to interact with people and get infected
link.springer.com/article/10.100…
13/
So one can exclude the following blood donor studies from Ioannidis' list of 61 studies in part 8/:
9 studies:
11, 14, 17, 29, 33, 35, 44, 46, 61
14/
Studies of leftover samples from hospital visitors are non-representative. For instance:
- one can get infected at a hospital (nosocomial infection)
- some visit hospitals due to concern they're infected with SARS-CoV-2
link.springer.com/article/10.100…
15/
So one can exclude the following 'residual sera / hospital visitors' studies from Ioannidis' list of 61 studies in part 8/:
13 studies:
2, 9, 23, 27, 30, 31, 36, 39, 40, 42, 47, 59, 60
academic.oup.com/cid/advance-ar…
academic.oup.com/cid/advance-ar…
16/
Volunteers who hear antibody testing is occurring, but were not targeted for testing by random selection, may ask to be tested because they think they're infected (ex: they have symptoms, known prior exposure to an infected person, etc.).
uoflnews.com/post/uofltoday…
17/
A similar problem occurs for studies with non-randomized (non-probabilistic) sampling steps after initial randomization.
Besides studies in part 18/, many of the other studies Ioannidis cites use non-targeted volunteers.
link.springer.com/article/10.100…
18/
So one can exclude the following 'non-targeted volunteers / non-probabilistic step' studies from Ioannidis' list of 61 studies in part 8/:
10 studies:
12, 13, 15, 34, 38, 48, 50-52, 54
rapidreviewscovid19.mitpress.mit.edu/pub/p6tto8hl/r…
19/
People in schools, workplaces, shoppers, etc. are not necessarily representative of the general population. For instance, they're socially interacting in a closed setting, increasing their risk of infection.
20/
So one can exclude the following 'workplaces / schools / shops' studies from Ioannidis' list of 61 studies in part 8/:
6 studies:
10, 16, 19, 20, 22, 58
ncbi.nlm.nih.gov/pmc/articles/P…
21/
So out of 61 studies Ioannidis cites in part 8/, only 23 have at least decently representative sampling:
1, 3-5, 7, 8, 18, 21, 24-26, 28, 32, 37, 41, 43, 45, 49, 53, 55-57
(note: part 13/ should include 10 studies, not 9, since paper #6 is also blood donor study).
22/
But papers 4, 5, 7, + 8 should be dropped from IFR calculations, since they sample regions included in paper 3.
That leaves 19 papers.
Sampling the same region multiple times unfairly skews the results, as per collinearity.
who.int/bulletin/onlin…
23/
Paper 18 should be dropped, since it implies only ~365 infections, which is much too small a number to derive a robust IFR estimate.
It's telling that Ioannidis tried to infer an IFR of 0.00% from that.
wwwnc.cdc.gov/eid/article/26…
24/
Re: "It's telling that Ioannidis tried to infer an IFR of 0.00% from that."
Ioannidis states it's risky to draw inferences from a sample size of less than 500... but doesn't let that stop him from using ~365 infections to infer a low IFR.
who.int/bulletin/onlin…
25/
Paper 37 (Karachi, Pakistan) should also be dropped, because Ioannidis uses an unsound method of getting the COVID-19 deaths he applies in his IFR calculation.
medrxiv.org/content/10.110…
aku.edu/news/Pages/New…
who.int/bulletin/onlin…
26/
That leaves 16 studies:
1, 3, 21, 24-26, 28, 32, 41, 43, 45, 49, 53, 55-57
(note: part 21/ should read say "only 22", not "only 23")
That's barely a quarter of the 61 studies Ioannidis said supported his IFR estimate. But at least the studies are now of better quality.
27/
Yet problems remain.
Take the example of paper 3 (for Brazil). Months ago when I first saw what Ioannidis did with paper 3, I lost any remaining confidence I had in the credibility of his paper and in his IFR work in general
who.int/bulletin/onlin…
28/
Paper 3 stated an IFR of ~1.0%.
If Ioannidis' paper was really a *systematic* review, then he would have used that IFR to start with, as he did for some other studies he cited.
Instead Ioannidis was non-systematic + biased.
medrxiv.org/content/10.110…
29/
To avoid paper 3's larger stated IFR, Ioannidis invented a new calculation based on his misinterpretation of figure 3 from the paper. In doing so, he contradicted paper 3 and used reasoning that makes no sense.
who.int/bulletin/onlin…
30/
Suppose a country of 10,001,000 people contains:
- city X: 10,000,000 people
- ten other towns: 100 people per town
Obviously, X predominately determines the country's IFR. But Ioannidis' method gives the same weight to X as to one of the towns. 🤦♂️
medrxiv.org/content/10.110…
31/
There are other IFR calculation issues. For example, Ioannidis gives an uncorrected IFR of 0.45% for paper 43 for Geneva (see image on the right side in part 8/).
But co-authors of paper 43 later used the paper's data to calculate an IFR of 0.64%:
thelancet.com/journals/lanin…
32/
Addressing those obvious issues for papers 3 and 43 leaves one with the following median IFR from the 16 studies in part 26/:
~0.5%
Similar median IFR @GidMK showed previously, but now with the removal of residual sera studies, collinearity, etc.
33/
This median estimate of ~0.5% would likely go up once one corrected other issues with Ioannidis's paper.
Some of these issues include:
- under-estimated COVID-19 deaths via right-censoring
thelancet.com/journals/lanin…
link.springer.com/article/10.100…
34/
- under-estimating deaths b/c of death reporting
ncbi.nlm.nih.gov/pmc/articles/P…
ncbi.nlm.nih.gov/pmc/articles/P…
ijmr.org.in/article.asp?is…
medrxiv.org/content/10.110…
medrxiv.org/content/10.110…
medrxiv.org/content/10.110…
bmj.com/content/370/bm…
sciencedirect.com/science/articl…
35/
- over-estimating seroprevalence for papers 21, 28, 49 (Gangelt, Guilan, Los Angeles), and papers 56 + 57 under-estimating seroprevalence by a smaller proportion
36/
Take-home messages:
- Ioannidis predominately uses studies with non-representative sampling
- excluding the most non-representative studies doubles his IFR to ~0.5%
- correcting his other mistakes would most likely increase his stated IFR even more
37/
Re: "doubles [Ioannidis'] IFR to ~0.5%"
Ironic.
"Ioannidis told viewers that the virus has an “infection fatality rate that is in the same ballpark as seasonal influenza.”"
buzzfeednews.com/article/stepha…
who.int/publications/m…
38/
The difference between an IFR of ~0.6% and IFR of ~0.2% is 3X less COVID-19 deaths (with the same number of infections).
Ex: the USA now has ~320,000 reported COVID-19 deaths. IFR 3X lower would be ~213,000 more people alive
39/
Alongside parts 5/ and 6/:
"[IFR in a typical low-income country:] 0.23% (0.14-0.42 [...]) [...].
[In a typical high income country:] 1.15% (0.78-1.79 [...])"
web.archive.org/web/2020110103…
From May, before move evidence on IFR:
[mdpi.com/2076-393X/8/2/…]
sciencedirect.com/science/articl…
40/
There are other oddities that individually wouldn't mean much, but taken together show Ioannidis is biased towards showing a lower IFR.
For example, he uses an 0.09% IFR for paper 26 in part 8/, when his source gives a higher value:
medrxiv.org/content/10.110…
41/
Similarly, Ioannidis gives an IFR on 0.28% for paper 21 in part 8/, even though his cited source gives a higher IFR.
In any event, his source likely under-estimates IFR by at least a factor of 4.
medrxiv.org/content/10.110…
42/
Ioannidis uses PCR results for paper 56, even though PCR-positive antibody-test-negative people would be less likely to die in time to be included in his IFR.
That allows him to reduce paper's 56's IFR from 1.67% to 0.91%.
medrxiv.org/content/10.110…
43/
And Ioannidis reports a 1.16% IFR for paper 45 from part 8/ (for England, without excluding nursing home deaths), while the source he cites gives an IFR of 1.43%.
medrxiv.org/content/10.110…
44/
Jay Bhattacharya uses Ioannidis' age-specific result to influence governments:
"For people 70 and over, the infection survival rate is 95%. For people under 70, it is 99.95%"
web.archive.org/web/2020120920…
^^^That doesn't hold up.
link.springer.com/article/10.100…
45/
Paper 37 has three phases of testing, occurring one after the other. Ioannidis knew of phases 1 and 2.
He left out phase 1, + gave phase 2 an IFR of 0.08%.
An update today rebuts that:
medrxiv.org/content/10.110…
46/
Some try to support Ioannidis' paper by cherry-picking studies with a lower population-wide IFR.
Ex: a ~0.3% IFR for Iceland.
47/
But population-wide IFR tends to be higher in:
- older countries
- places with a greater proportion of infections in older people
Accounting for that reveals an age-specific IFR higher than in Ioannidis' paper (see part 44/)
nejm.org/doi/suppl/10.1…
48/
The WHO publishing Ioannidis' paper in their journal doesn't mean they agree with him (ex: a journal can have 2 papers that contradict each other)
WHO officials endorse an IFR estimate >2X larger than Ioannidis' (see part 7/)
who.int/bulletin/discl…
49/
Below is a thread that uses the same method to analyze 4 other seroprevalence-based multi-country IFR estimates:
50/
The population fatality rate, or PFR (i.e. COVID-19 deaths per capita), should be less than IFR, since you can't have more people infected than actually exist.
Yet in at least 3 instances, Ioannidis gives 'corrected' IFRs that are less than PFR.
51/
Re: "Ioannidis gives 'corrected' IFRs that are less than PFR"
For the paper #'s in part 8/, with IFR listed before PFR:
#6
0.11% , 0.19%
transparencia.registrocivil.org.br/especial-covid
#46
0.07% , 0.15%
coronavirus.data.gov.uk/details/deaths…
#50
0.00% , 0.04%
coronavirus.jhu.edu/us-map
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.