Health Nerd Profile picture
Oct 15, 2021 25 tweets 7 min read
This paper came out recently, and it is HUGELY popular among anti-vaccine advocates who are using it to suggest that vaccines don't work

It's also very fundamentally flawed. I'm a bit surprised it was published. Some thoughts 1/n Image
2/n The paper is here, and it's basically a series of comparisons of publicly-available COVID case and vaccine data conducted by a Harvard professor and a high-school student (note-DO NOT BE MEAN TO THE STUDENT, NEVER BE MEAN TO STUDENTS)… Image
3/n The study is broken into 2 sections. In the first, the authors took @OurWorldInData info, comparing the previous 7 days of case data between countries by vaccine rates. They produced this graph, showing no relationship between vaccines and reported cases Image
4/n There are several fundamental mistakes with this analysis. The first one is a biggie - confirmed cases, as reported by countries, are a useless statistic by themselves
5/n This goes back to the very early days of the pandemic - cases are, by definition, a function of tests. Without tests, you don't confirm cases, and therefore this comparison makes no sense at all on its own
6/n Did this impact the author's graph?

Very clearly yes. For example, let's compare the country Georgia (5408 cases/mil) with some of the places with less than 50 cases/mil in the author's graph on case/testing numbers ImageImage
7/n So the y-axis of this figure is uninterpretable. Meaningless.

What about the x-axis? Image
8/n There are 2 ways you can become immune to COVID-19 - vaccines and PRIOR INFECTION

In some of these countries (India, Russia, USA etc) it's likely that upwards of 40% of the entire population is immune to COVID-19 because they've had the disease and recovered (or died)
9/n In other words, we might expect that in places like Brazil, where 1 in every ~350 people has died from COVID-19 since the pandemic began, there would be quite a bit of immunity even with relatively low vaccination rates
9.5/n So comparing places on case numbers without taking infection-induced immunity into account is nonsensical, unless for some reason you think that it does not prevent infections
10/n On top of this, we've got the issue of an arbitrary 7-day window. Angola looked really good if you downloaded the data on 3/09. It looks less good if you download the data today Image
11/n So the first graph is pretty much useless as evidence. What about the second bit of the analysis?

Well, the authors essentially did the same thing, but for US counties. They found no difference in cases in US counties by vaccination status ImageImage
12/n So vaccines didn't prevent cases in the US!

Except, well, there's issues here too. Let's look at the places that the authors explicitly mention in the text Image
13/n Going to the file that the authors used, from Healthdata dot gov, you can see some really weird things pop up. Chattahoochee, GA, has a population of 10,907, but has 30,233 people who are fully immunized Image
14/n Indeed, looking down the list, all three of the counties identified by name - Chatahoochee, Mckinley, and Arecibo - have vaccinated more people than live there by quite a wide margin Image
15/n Meanwhile, of the counties classified as "low" transmission have rates of vaccination not just below 20%, but close to 0%!

What's going on here? Image
16/n The answer appears, based on this excellent @jburnmurdoch thread, to be that in the US vaccine doses are recorded based on the county *where they are given* not where the person lives
17/n This actually explains those discrepancies very neatly. Arecibo Municipio, for example, contains a stadium listed as a mass vaccination site in Puerto Rico Image
18/n On top of this, we've got the same issue with picking an arbitrary 7-day window as before - the median population in a county from this spreadsheet is ~26k people, which means that small variations in case numbers make a big difference
19/n This is even more apparent when looking at those "low" transmission counties - the median population is ~1k people, with some counties having <100. Weekly variation makes a HUGE difference!
20/n For example, Mineral County, CO, pop 769 is a "low" transmission county on 9/2, but a week later on 9/9 it's a "high" transmission county because it reported...2 cases
21/n All in all, this analysis is a bit meaningless as well. Using a 7-day period is pointless, it again ignores infection-derived immunity, and the vaccine rates are clearly not correct by county
22/n Ultimately, I don't think this paper has much, if any, meaning, and it's kind of bizarre that it was published at all. It certainly proves nothing whatsoever about vaccines and their effectiveness
23/n Oh, also, these are the most basic issues in the paper. I didn't even get started on things like Non-Pharmaceutical Interventions, confounding, etc etc etc
24/n Worth noting that the authors' intentions were clearly not anti-vaccine per se, but I do think that the paper is nevertheless not sufficiently robust to actually conclude anything about vaccines Image

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Health Nerd

Health Nerd Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @GidMK

Feb 3
A new study is out on Vitamin D and COVID-19, and has reported a benefit, with massive acclaim from the usual suspects

It is, however, one of the worst papers I've ever seen. So many mistakes that it's hard to know where to start 1/n Image
2/n The paper is here. It is a meta-analysis of D for COVID-19 that aggregated together just 5 studies and concluded...this:… Image
3/n This is a huge claim, given that the largest, most well-controlled studies of D for COVID-19 have not found any benefits. How do the authors come to this conclusion?…
Read 16 tweets
Jan 23
Here's the thing - most people who say they don't value the opinions of experts are talking utter bullshit

They just dislike expertise as it relates to their political opinions
People will dismiss the opinions of experts in, say, climate science, but they don't go to some random dude Jim's garage to get their engine checked out. Not many people lining up for amateur accountants to do their taxes
Even in health the same holds true. People get really dismissive about clear facts of public health, because they've been shown by experts, but you don't see many people getting hip replacements from non-experts!
Read 4 tweets
Jan 14
Nuremberg 2.0 is still trending, which only proves just how ignorant so many people are about the Holocaust and Nuremberg trials
Hate to break it to you but doctors giving advice does not come even vaguely close to the horrors revealed at the Nuremberg trials EVEN IF IT IS ADVICE YOU DO NOT PERSONALLY AGREE WITH
Lots of people replying to this who didn't bother to look up the Nuremberg trials before getting all huffed up and self-important online
Read 4 tweets
Jan 13
Interesting paper on Long COVID. Using similar methodology to previous research, but on a mild and relatively low-risk population, researchers found that long-term issues were uncommon and mostly resolved within 12 months…
This paper explicitly excluded people who went on to be hospitalized for COVID-19, and as you can see from the age breakdown looked at a very low-risk cohort generally
Basically, they found that people with mild disease in this dataset who tested positive for COVID-19 had a moderately increased risk of various negative outcomes, which declined substantially over time
Read 7 tweets
Jan 13
Interestingly, this is almost universally untrue. The trope of "lone genius" doesn't even really hold up to scrutiny for historical scientists, never mind people working in science today
I mean, take Newton. Great history of science, discovered gravity, apple on the head etc

Except, Newton built his ideas on centuries of prior research, and may have stolen his greatest ideas from someone else
Point is, even if Isaac Newton didn't steal ideas from Robert Hooke, he definitely spent nearly a decade working with the man. And both Hooke and Newtown were influenced by Descartes, through English scientists, who in turn used work done before his time as well
Read 6 tweets
Jan 12
This new "gas stoves cause 12.7% of childhood asthma" thing is a great example of how we often aren't very good at communicating uncertainty

I don't really agree with these headlines 1/n
2/n The headlines are based on this study, which is a very simple paper using a previous meta-analysis to calculate what's called a Population Attributable Fraction of current asthma associated with gas cooking…
3/n The formula for PAF is extremely simple, here's what these authors used. They took the estimates for RR (technically OR) from this previous meta-analysis…
Read 16 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!