Thread by @AtomsksSanakan

1/J

John Ioannidis published an article defending his low estimate of COVID-19's fatality rate.

It contains so many distortions that I'll try something I've never done on Twitter for a paper:

Go thru distortions page-by-page.

This will take awhile. 😑

onlinelibrary.wiley.com/doi/10.1111/ec…

2/J

Some context:

Infection fatality rate, or IFR, is the proportion of people infected with the virus SARS-CoV-2 who die of the disease COVID-19.

There are many IFR estimates, including some from Ioannidis.



institutefordiseasemodeling.github.io/nCoV-public/an…

3/J

Seroprevalence studies (serosurveys) measure antibody levels to estimate the number of infected people.

Dividing COVID-19 deaths by that number of infected people gives a seroprevalence-based IFR.



who.int/bulletin/volum…

4/J

One can over-estimate seroprevalence (+ thus under-estimate IFR) by measuring seroprevalence in a sample that does not represent the general population, and then extrapolating that sample to the general population.

Ioannidis does this.

5/J

Ioannidis defends his use of non-representative samples.

But his defense fails. For example:

- non-representative samples are still unreliable
- he uses non-representative samples even in hard hit areas



who.int/bulletin/volum…

6/J

- he could just wait for representative sampling in less hard hit areas
- areas often looked less hard hit because they under-estimated COVID-19 deaths, so including them under-estimates IFR

etc.

bmj.com/content/370/bm…



medrxiv.org/content/10.110…

7/J

So in this thread, *keep this in mind*:

Ioannidis has to keep non-representative samples in, because representative samples show an IFR incompatible with his position.

That's his main game, + what he often distracts from

8/J

With that framework in place, let's start with the page-by-page review of Ioannidis' paper:

Ioannidis excludes @GidMK + @BillHanage's paper Levin et al., because it focused on specific countries.



link.springer.com/article/10.100…

onlinelibrary.wiley.com/doi/10.1111/ec…

9/J

Ioannidis' exclusion fits with him under-estimating IFR by using non-representative samples in areas that under-estimate COVID deaths.

The WHO + the USA's CDC know better, and so rely on Levin et al.:

web.archive.org/web/2021032419…


link.springer.com/article/10.100…

10/J

Seroprevalence-based IFR was ~0.76% in @LeaMerone + @GidMK's paper, when they focused on seroprevalence studies with a low risk of bias.

Ioannidis conveniently leaves that out.

sciencedirect.com/science/articl…


onlinelibrary.wiley.com/doi/10.1111/ec…

11/J

Ioannidis co-authored 2009 PRISMA guidelines that stated one should competently assess studies for risk of bias.

@LeaMerone + @GidMK did that.
Ioannidis didn't, letting in non-representative samples

bmj.com/content/339/bm…

12/J

The "low IFR" Ioannidis references is one he inferred from a Los Angeles County study.

That IFR is impossible since it requires more people are infected than actually exist.

jamanetwork.com/journals/jama/…


onlinelibrary.wiley.com/doi/10.1111/ec…

13/J

The "IFR = 0.31%" study Ioannidis mentioned is below.

@LeaMerone + @GidMK excluded it because "did not allow for an estimate of confidence bounds"
sciencedirect.com/science/articl…

"to estimate an overall IFR for the United States of 0.863 percent"
papers.ssrn.com/sol3/papers.cf…

14/J

It's 0.31% IFR is unreliable anyway since, for example, the studies for Santa Clara, New York (both), + Chelsea used non-representative sampling. Miami-Dade was wrong.

ncbi.nlm.nih.gov/pmc/articles/P…

15/J

ICCRT:
imperial.ac.uk/mrc-global-inf…


- Ioannidis' IFRs for LA County + Scotland are impossible:



- Gangelt over-estimated the seroprevalence:


onlinelibrary.wiley.com/doi/10.1111/ec…

16/J

- Kenya used non-representative sampling on blood donors
science.sciencemag.org/content/371/65…


- Due to co-linearity, the nationwide study ICCRT cites supplants Rio Grande do Sul
nature.com/articles/s4159…



imperial.ac.uk/media/imperial…

17/J

Taking a break for a bit.

The thread so far covers *less than a page* of the distortions + misleading statements in Ioannidis' paper.

I hope people understand why many experts in this field no longer invest time in addressing his nonsensical under-estimating of IFR. 🤦‍♂️

18/J

Why the following studies were non-representative:

- Luxembourg: non-probabilistic selection step

page 6: medrxiv.org/content/10.110…

- New York: sampled shoppers
ncbi.nlm.nih.gov/pmc/articles/P…


onlinelibrary.wiley.com/doi/10.1111/ec…

19/J

- the New York sample under-estimated IFR (see 18/J)

- low response rate biases seroprevalence up, under-estimating IFR


- the IFR in Italy was likely over-estimated, due to lower sensitivity of the Abbott assay

onlinelibrary.wiley.com/doi/10.1111/ec…

20/J

His adjustment makes no sense since it's already implicit in test adjustments for sensitivity.

And IgA assessment isn't required, given IgG.
thelancet.com/action/showPdf… (table 2)
ncbi.nlm.nih.gov/pmc/articles/P…
bmj.com/content/370/bm…



onlinelibrary.wiley.com/doi/10.1111/ec…

21/J

I'll leave to others (maybe @GidMK?) to discuss the meta-analysis details.

But I can say Ioannidis under-estimates seroprevalence-based IFR in southeast Asian countries such as Japan + South Korea.




onlinelibrary.wiley.com/doi/10.1111/ec…

22/J

There are at least three approaches to dealing with areas lacking representative samples:

1) exclude those areas + wait for data
2) use regions with representative samples to extrapolate over
3) include non-representative samples from those areas

onlinelibrary.wiley.com/doi/10.1111/ec…

23/J

#3 is worst because it extrapolates from inaccurate samples, under-estimating IFR. Yet that's what Ioannidis chooses to do + uses Bobrovitz for.

#1 makes sense; that's what "Meyerowitz-Katz" (@GidMK) did. But if you must have data for policy or planning, #2 can work.

24/J

And now in his discussion section, Ioannidis turns to the core point.

I'll spend a few tweets on this because this is *the* central pillar of his position, and is how he's been misleading millions of people for over a year.



onlinelibrary.wiley.com/doi/10.1111/ec…

25/J

Suppose you want to know what proportion of people in a city like dogs.

You could survey people in 1 building.

By luck the percentage you get might match the percentage you would get for the city overall. But you didn't design the survey to make that more likely.

26/J

The same point applies to seroprevalence studies.

Non-representative sampling might *luckily* get results that match the overall population. But representative sampling is *designed* to be more likely to match the population.

academic.oup.com/cid/advance-ar…

27/J

Scientists know methods that get representative samples that are more likely to match the general population; they applied them to diseases before COVID-19.

Ioannidis discards those methods, + relies on non-representative sampling luckily matching.

28/J

So his "[n]o consensus" claim is misleading. There's an evidence-based consensus (outside of Ioannidis) that those samples could *luckily* match, but are not designed to + are thus less likely to.

Covered in another thread:



onlinelibrary.wiley.com/doi/10.1111/ec…

29/J

And "non-participating invitees" are less likely to be infected, so Ioannidis was wrong. We don't the response rate for his Santa Clara study, since he has no targeted sample.


medrxiv.org/content/10.110…



medrxiv.org/content/10.110…

30/J

Most infected people increase antibody levels. In the general population that antibody increase persists for ≥6 months in most people, besides with some assays like Abbott.





onlinelibrary.wiley.com/doi/10.1111/ec…

31/J

Unlikely policies caused more excess deaths; non-COVID-19 deaths dropped.

sciencedirect.com/science/articl…
academic.oup.com/aje/advance-ar…
medrxiv.org/content/10.110…
bloomberg.com/opinion/articl…





onlinelibrary.wiley.com/doi/10.1111/ec…

32/J

- Germany's excess deaths for the 1st wave were low in simple comparisons to previous years
archive.is/0oQH8

- scientists make the necessary adjustments already


- the Germany study Ioannidis cites undermines his claim
ncbi.nlm.nih.gov/pmc/articles/P…

33/J

Levin et al. focused on studies where death numbers were not accelerating after the seroprevalence study. That helped mitigate over-estimation of deaths
link.springer.com/article/10.100…

Developing countries can have reporting delays
thelancet.com/journals/lance…

onlinelibrary.wiley.com/doi/10.1111/ec…

34/J

The Brazil "1%" IFR study is peer-reviewed, as are others
thelancet.com/journals/langl…


#12 overlaps with 0% seroprevalence 🤦‍♂️
nature.com/articles/s4159…

#54 is non-representative sampling of blood donors
science.sciencemag.org/content/371/65…

35/J

IFR with representative sampling is higher in Japan:


With the possible exception of #58, his cited studies use non-representative sampling:


#58 contradicts his low IFR:


onlinelibrary.wiley.com/doi/10.1111/ec…

36/J

He complains about having to follow the PRISMA guidelines he co-authored 🙄

IFR increased with less risk of bias (as expected when non-representative sampling skews IFR down):
sciencedirect.com/science/articl…




onlinelibrary.wiley.com/doi/10.1111/ec…

37/J

It's telling how Ioannidis twists O'Driscoll et al. to support a lower IFR that contradicts their paper:



Same for Ioannidis returning to abusing non-representative samples:


onlinelibrary.wiley.com/doi/10.1111/ec…

38/J

His IFR in Europe is an under-estimate:
thelancet.com/journals/lance…
wwwnc.cdc.gov/eid/article/27…
sciencedirect.com/science/articl…
link.springer.com/article/10.100…

Extends to Andorra (~0.6% IFR):
covid19.who.int/region/euro/co…
researchsquare.com/article/rs-119…

And on the Faroe Islands:

39/J

Now, some people might wonder why I spent so much time explaining factual errors in this paper.

Well, it's wrong and needs correction. But there's another reason:

It's the most unprofessional + disingenuous peer-reviewed paper I've ever read.

40/J

Re: "It's the most unprofessional + disingenuous peer-reviewed paper I've ever read."

If you don't believe me on that, then read Appendix 1 on page 38, and ask yourself if you've *ever* seen this in a scientific paper from a competent scientist:

onlinelibrary.wiley.com/doi/10.1111/ec…

41/J

I have seen scientists address criticism of their work from Twitter, usually by responding to the general criticism without mentioning the tweets.

I have *never* seen a scientist evade criticisms of their work, while naming Twitter accounts, their number of tweets, etc.

42/J

In doing this, Ioannidis has tacitly admitted he can't rebut criticism of his work.

Are any of the people who whined about negative tone of Ioannidis' critics going to call him out for this?

Of course not. 🙂



onlinelibrary.wiley.com/doi/10.1111/ec…

43/J

And unsurprisingly, Ioannidis extends his evidence-free ideologically-motivated smears to Ferguson et al.'s team at Imperial College, without ever admitting they did not over-estimate IFR.



onlinelibrary.wiley.com/doi/10.1111/ec…

44/J

So for Ioannidis' defenders, that's what you're left with:

An ideologue who makes mathematically impossible claims that suit his ideology, and names Twitter accounts in his paper since he can't rebut their critiques.

🤷‍♂️



archive.is/dT97F#selectio…

45/J

And there's Ioannidis' not so subtle attempt to encourage his fans to dox me.
en.wikipedia.org/wiki/Doxing

Are any of the people who incorrectly whined about Ioannidis (supposedly) being "silenced", going to call him on that?

Of course not. 😑

onlinelibrary.wiley.com/doi/epdf/10.11…

46/J

Also, this was not a one-time lapse in judgment on Ioannidis' part. He's done this before in the same journal.

So Ioannidis was editor-in-chief at European Journal of Clinical Investigation for a decade, and it's now his venue for attacking people

47/J

And gracious threads from the researchers in question:




sciencedirect.com/science/articl…

48/J

Found the PubPeer thread on Ioannidis' paper, for those who want to defend it or critique it (I don't post on PubPeer, nor do I have an account there):

pubpeer.com/publications/9…