Thread by @davidwdowdy

The big COVID news, dropped on a Fri afternoon:
"effectiveness vs COVID-19 hospitalization was higher for Moderna than Pfizer (and Janssen/J&J)."

bit.ly/3kjuoFR

But let's take a look at the analysis before accepting these results at face value...

1. Here are the data - on the surface, looks like Moderna effectiveness is constant at 92-93% after 120 days, Pfizer drops from 91% to 77%.

(btw, this was CDC not Pfizer funded, but what a great strategy for Pfizer - push "waning immunity" so people need a 3rd dose of your vax)

2. But note that the comparison group for all of these is the same - difference b/w numerator & denominator is always 1463 cases & 899 controls.

Meaning that people w Moderna & Pfizer (& J&J) are being compared to the same unvax'd folks.

3. Two implications. First, sample size.

Since unvax'd people are the same, relative effectiveness comes down to # of vax'd cases.

Comparing <120d vs >120d, Moderna had 36 vs 18, Pfizer 65 vs 63.

If 9 people w Moderna got sick >120d instead of <120d, conclusions change.

4. Second, geographic region.

43% of Pfizer given in South, 26% of Moderna.

If exposure levels were higher in the South (likely), and vax is less effective vs intense exposure, Pfizer looks worse.

Results were adjusted for region, but wouldn't remove this effect modification.

5. Even worse, exposure by region was changing over time. (See data on cases in kids from bit.ly/2ZbLyNJ.)

So, if vax is least effective where exposure is most intense, Pfizer immunity could look like it's waning - just b/c in the South, exposure was rising (Mar-Aug).

Bottom line:
Both vaccines work well. But take Moderna-Pfizer comparison w a grain of salt.

1. Effective sample size is small.
2. There's a real risk of bias due to different exposure levels.

These data probably have some truth, but aren't as solid as they might first appear.