Thread by @CarldeBoerPhD

Not in the habit of COVID paper takedowns, but I figured my investigation might be useful to others too. The paper making the antivax rounds is: "Increases in COVID-19 are unrelated to levels of vaccination across 68 countries and 2947 counties in the United States"

The paper is rife with mis-citation (impressive, given there are only 10 references). Here are some of the more egregious examples:

Eg 1: "For instance, in a report released from the Ministry of Health in Israel, the effectiveness of 2 doses of the BNT162b2 (Pfizer-BioNTech) vaccine against preventing COVID-19 infection was reported to be 39% [6], substantially lower than the trial efficacy of 96% [7]. "

Looking at Ref 7 (ncbi.nlm.nih.gov/pmc/articles/P…), the 96% figure appears to be "Vaccine efficacy against severe disease was 96.7% (95% CI, 80.3 to 99.9)." which, as I'm sure you noticed is NOT "preventing COVID-19 infection".

A closer look at Ref 6 (gov.il/BlobFolder/rep…), reveals that the 39% protection from infection IS protection from infection, and that the same data show a 91% protection from severe disease:

"Even though vaccinations offers protection to individuals against severe hospitalization and death, the CDC reported an increase from 0.01 to 9% and 0 to 15.1% (between Jan-May 2021) in the rates of hospitalizations and deaths, respectively, amongst the fully vaccinated [10]."

BUT THIS IS NOT WHAT Ref 10 (context-cdn.washingtonpost.com/notes/prod/def…. <- note period in URL) SHOWS (slide 4):

It shows is the rate of fully vaccinated among hospitalized people (and deaths) - NOT the rates of hospitalization and deaths among those vaccinated. VERY different things.

The text reads as if the vaccines are losing their effect, when in fact it reflects an increase in the % of people who are vaccinated (WHICH IT SAYS ON THE SLIDE THEY CITED).

Now to the central finding of the paper (shoutout to @skepticalraptor for their post on this paper which influenced my thinking): skepticalraptor.com/skepticalrapto…

Here it is: 🥁🥁🥁🥁🥁

They found a minimal correlation between COVID cases/million (IN THE LAST 7 DAYS) and % fully vaccinated. There are a few pretty major issues with this.

Issue 1: Why are they looking at covid cases in the last 7 days? Why not look across time within each country?

Issue 2 (and it's a big one): There was no effort to control for confounders. There's two big potential confounders I can see here: Delayed reporting, and money

Delayed reporting: Sometimes there's a lag between COVID case numbers reported and the actual case numbers observed that week. How much does delayed reporting affect this? How much does the delay in reporting vary by country?

Money: Not every country is testing to the same degree. Not every country is vaccinated to the same degree. Vaccines and testing costs money. Now for some data from @OurWorldInData !

Sure enough, we see more cases in richer countries, but we also see more testing in richer countries:

So what's the relationship between GDP and vaccine rates? Here's a plot from July 2021 from Nature (nature.com/articles/d4158…). Richer countries are more vaccinated:

Given this complex relationship between GDP and vaccine status, testing rates, and COVID cases, it's not clear what this paper's central finding means.

So what does this paper show? Not much. I certainly would not use it as a reason not to use vaccines to curb COVID19 spread. Honestly, this was a pretty big let down from @SpringerNature, and I hope they will do something about it.

Happy and grateful to get feedback on this from experts in the field, as I myself am not one. Thanks for reading, and I hope some may find this useful!
@EricTopol @AnnaBlakney @CaulfieldTim

Update: another thing that people (eg me) seem to have overlooked: The paper says: "Across the US counties too, the median new COVID-19 cases per 100,000 people in the last 7 days is largely similar across the categories of percent population fully vaccinated (Fig. 2). "

BUT there is a clear relationship between vaccination rate and COVID cases in the last 7 days! Counties with >55% have <200 cases/100k (median), while counties with <30% vaccination have ~500 cases/day.

So their conclusions are not even supported by their own analysis. It's still a poor analysis, so I wouldn't use Fig 2 in support of vaccines, even though the trend it shows is in support of their ability to protect against infection.