Thread by @GidMK

This paper was recently published, arguing that vaccines cause as much death as they prevent and so we should stop vaccinating people

I rarely say this, but it is truly awful and should be retracted as soon as possible 1/n

2/n The paper is here. It is truly woeful, but worth reading just to see how easy it can be to make a plausible-sounding argument if you are very free with your methodology mdpi.com/2076-393X/9/7/…

3/n The authors did two things - they calculated a Number Needed To Vaccinate (NNTV) from a propensity-matched cohort study done in Israel. They also calculated the number of deaths reported through the Dutch vaccine reporting system

4/n They then compared these two numbers, arguing that since the NNTV was almost equal to the number of deaths reported after vaccination in the Netherlands, vaccinations are not a good intervention

5/n The first obvious issue here is in the NNTV

It is not a great statistic, but here it is used in a WILDLY misleading way

6/n NNTV is easily calculated - you just divide 1 by the absolute risk difference between the vaccinated and unvaccinated groups. In the Israeli study, the risk of death was 0.006% higher in unvaccinated people, therefore the NNTV was 1/0.00006 = 16,667

7/n But here's the thing - this trial was only 6 weeks long. Fewer than 3% of the total population got COVID-19 in that time, compared to at least 30% of the entire of Israel over the last 12 months

8/n This means that the NNTV from this study is INHERENTLY MISLEADING unless you assume that vaccines will stop working entirely after the 6-week period (obviously false)

9/n If you extrapolate this efficacy out linearly, and assume that the RELATIVE risk of death after vaccination remains similar over time, at 52 weeks you'd get an NNTV of 1/((0.00006/6)*52) = 1 per 1,960

10/n If you assume that everyone who stays alive will get infected without a vaccine eventually - which is a fact - the ABSOLUTE risk difference approaches the RELATIVE risk difference, and so NNTV = 1/0.84 = 1.2

I.e. 1 life saved for every 1.2 vaccines given!

11/n (Note - this, too is misleading. The NNTV as time trends towards infinity in this population is high because they were older and sicker than the total population. For example, the NNTV for 10-year-olds will be 1,000 times higher than that for 60-year-olds)

12/n On the other side, the paper uses a vaccine adverse event reporting system to estimate the number of deaths 'caused' by the vaccine

This is absolute gibberish and a basic misunderstanding of epidemiology

13/n Vaccine event reporting systems are geared to identifying potential signals for alarm, and so usually anyone can report any event THAT HAPPENS AFTER VACCINATION to them

14/n In other words, these are events FOLLOWING vaccination, not events CAUSED BY vaccination

The website of the system even says this ~explicitly~

15/n So while the Israeli study was an attempt to causally link vaccination with outcomes, the Dutch registry explicitly does not do this

The deaths in this system may have nothing to do with the immunizations at all

16/n Using reporting systems like this is a common anti-vaccine trope. We KNOW that many of the events in the reporting system ARE NOT LINKED TO VACCINATION because we investigate them carefully

17/n Because of this, we have two awful, useless numbers being compared to each other. The true rate of deaths CAUSED by vaccines is 100sx lower than this paper calculates, and the number of deaths PREVENTED is 100sx higher

18/n Taking the highest death rate seen caused by a vaccine so far - Astrazeneca - we'd get around 1 death per 500,000 vaccinations

Using an NNTV assuming a 30% population prevalence, we'd get 1 life saved per ~500 vaccinations

19/n Now, neither of these metrics are necessarily useful in this way, and people will yell at me for doing this (rightly, it's a bit silly), but EVEN USING THIS FLAWED RUBRIC vaccines save 1,000x more lives than they cost

20/n Anyway, this study is flawed in so many basic ways that it's pretty irretrievable. It should be retracted as soon as possible to avoid further damage

21/n Another note, H/T to @ScottinVictoria - the reviewer comments are pretty astonishing to read. Not even a mention of why NNTV is worthless in this context, and no argument about the manifestly wrong use of adverse event reports mdpi.com/2076-393X/9/7/…

22/n Also worth pointing out that two members of the editorial board of the journal have resigned so far because of this terrible study

https://twitter.com/ProfKatieEwer/status/1409125241142513670?s=20

23/n also as a couple of people have pointed out, this tweet is incorrect. I mixed up the ARR of disease prevention with death, in actuality the ARR would approach the death rate in the population x 0.84 - in most places this would be about 1 per 100/200

https://twitter.com/GidMK/status/1409293104063029252?s=19

24/n Oh, and because it always comes up in these discussions, I've never been paid a cent by any pharmaceutical company, all of my funding is through the Australian state and federal governments, the only additional income I get is from writing locked posts on Medium

25/n There is now an expression of concern published by the journal about the paper mdpi.com/2076-393X/9/7/…

26/n An interesting point about this paper is that it's actually a perfect example of how peer-review can fail. The people who reviewed the article assumed that the stats and methodology were reasonable, and based on that assumption recommended that it be published

27/n This is a prime example of what @jamesheathers and I wrote about recently - the study got through peer review, may be retracted, but the damage has already very much been done statnews.com/2021/06/08/sci…