Thread by @GidMK

Ok, so Ioannidis has updated his preprint

This is good - hats off to him for acknowledging issues and working to improve

Let's see what's changed 1/n

2/n The update is here: medrxiv.org/content/10.110…

3/n If you missed it, my original thread on this preprint is here:

https://twitter.com/GidMK/status/1262956011872280577?s=20

4/n So, reading the paper, most of the issues are still apparent:

- no clear search methodology
- strange inclusion/exclusion criteria
- odd 'adjustments' that only ever decrease IFR
- including strange studies
- excluding the most robust estimates

5/n What has changed is that some of these things are now justified

This is good, but not really adequate to improve the rigor of the paper

6/n For example, we now have this explanation of using the estimates derived from the included papers even if they didn't account for right-censoring

The thing is, the issue still remains, we now just have a few words addressing it

7/n There are also still obvious errors in the paper. Ioannidis claims that the Spanish seroprevalence estimate cannot be included because it has only been published as a press release

This is wrong

8/n In fact, the Spanish seroprevalence estimate is a lengthy government report that is FAR more detailed than many preprints. Excluding it from the main estimate makes no sense scientifically

9/n Similarly, the Czech Republic seroprevalence study is (while published in Czech) very comprehensive. The same is true of the Danish and English estimates (although not of the Swedish and Slovenian ones)

10/n In fact, it appears that Ioannidis has continued to exclude any government reports, which is still an issue (remember, governments are doing most of the testing!)

11/n So, the results

Broadly speaking, we have a bunch of new studies included but the exact same issues as before

12/n Studies with inappropriate samples to infer population IFR (such as the Kobe study) are still in there, while random, population-wide estimates (i.e. Spain) are excluded

13/n It's also worth noting that Ioannidis has violated his own inclusion criteria, with at least one study under the arbitrary 500-person sample size that has been included

14/n While it is hard to know why this is still the case, again the decisions made in the paper exclusively work to suggest a lower IFR than that actually implied by most research, which is worrying

15/n If we again only look at studies using a population-wide estimate of IFR, we see that the lowest estimate is still Ioannidis' Santa Clara study, with the estimates ranging from 0.18%-0.78%

16/n This is still a bit low - for some reason, this paper uses an incorrect IFR for the Brazilian estimate (0.3% instead 1% given by the authors) - but much more in line with the estimate from our updated meta-analysis of 0.64%
medrxiv.org/content/10.110…

17/n One thing worth noting - the paper still makes the clear error in comparing the IFR of COVID-19 to influenza

This is a common mistake, so I thought I'd highlight it

18/n Here, Ioannidis is comparing the IFR of influenza used by the CDC - which is ~0.1% - to the IFR of COVID-19 inferred from seroprevalence studies

These two figures, however, are not comparable

19/n The IFR estimate for influenza generated by the CDC is the result of a complex modelling process that inflates the numerator (deaths) according to hospitalization data for pneumonia and other ICD codes

20/n Why is this a problem?

Well, we are not comparing apples with apples here. Numerous efforts have demonstrated that the death count of COVID-19 in many places is a significant underestimate (by 50%+)

21/n If we instead compare the IFR of influenza calculated from seroprevalence studies and official death counts to the same for COVID-19, we see a VERY different picture

22/n The HIGHEST IFR estimate for influenza using this methodology, based on a 2014 systematic review, is 0.01%

That's 18x lower than the lowest reasonable estimate of COVID-19 IFR

ncbi.nlm.nih.gov/pmc/articles/P…

23/n More broadly, if we look at the total range, the IFR of COVID-19 calculated from seroprevalence data appears to be around 50-100x higher than the same number for influenza

24/n This is actually a serious flaw with the paper - the author has chosen only to pursue corrections of the data that push the IFR lower. If we were to account for excess mortality attributable to COVID-19 - based on published research - the IFRs would all jump substantially

25/n Now, there are some excellent improvements to the paper

For example, much of the language in the discussion/conclusion has been corrected

26/n There are still odd, emotive phrases ("blind lockdown"), but the paper no longer describes COVID-19 as common and mild, which was clearly incorrect

27/n However, overall this paper still suffers from many of the issues I previously raised, and seems to still substantially underestimate the IFR of COVID-19

28/n I should be clear that I am not speculating in any way about the reasoning behind these decisions. The fact that the paper underestimates IFR is a problem, but we can't really know why these decisions were made