Thread by @jsm2334

How well are vaccines and boosters really protecting against COVID-19 deaths?

Israel MoH publicly posted daily COVID-19 death data split by unvaccinated, boosted, and vaccinated-not-boosted from Aug10-Sept8

Here are results of my analysis of these data
1/n

Summing over all days, it is not promising to see so many COVID-19 deaths in vaccinated/boosted groups.
But by now we know better than to draw conclusions from raw counts, right?
2/n

The Israeli MoH dashboard provides enough information to infer total proportion of population unvaccinated, boosted, or vaccinated-by-not-boosted, so we can compute normalized COVID-19 death rates in these groups.

3/n

From these, we can compute overall estimates of vaccine/booster effectiveness(VE) in preventing COVID-19 deaths relative to unvaccinated.

These overall VE estimates are VERY low; some might infer from this the vaccines and boosters are NOT protecting vs. COVID-19 deaths.
4/n

But don’t forget Simpson’s Paradox.

We see much lower vaccination/booster rates in the young, who also have MUCH lower rate of COVID-19 death.

This confounding of age might distort the overall VE estimates, so let’s compute separate estiamtes for each age group.
5/n

VE estimates by age are much higher than overall, showing indeed there was a strong Simpson effect.

Before interpreting these results, however, we need to consider one other thing: Do the <60yr data include children and, if so, should we pull them out into their own group?
6/n

Given the total counts for all ages sum to >9.1m, the total Israeli population, it is clear children are among the <60yr group, and we can infer the number <12yr from the MoH data.

They are unvaccinated, and unlikely to comprise any of the 23 COVID-19 deaths.
7/n

Splitting children out dramatically affects the VE estimates for the <60 group as well as overall.

It is CRUCIAL to separate children out when looking at vaccine effectiveness, since they cannot be vaccinated and have the lowest risk of advanced COVID-19 events.
8/n

Now consider the results.

For older group, we see vaccines reduced risk of COVID-19 deaths during this time period without boosters, but boosters clearly increased level of protection.

This is consistent with published papers showing early results from boosting.
9/n

For younger group, we see high protection vs. COVID-19 deaths from vaccines, and in these data no evidence boosters improve protection.

However, given only 7% boosted and not much time passed, it is possible that future data will show a booster benefit for young adults.

10/n

Repeating this analysis on “critical COVID-19 disease”, we see similar results.

The vaccines protect against critical disease, with older adults showing potential waning of efficacy improved by boosters, & younger adults showing strong protection irrespective of booster.

11/n

Two key caveats in these data:
1. It is not clear how this MoH data set counted individuals partially vaccinated with a single dose.
2. Also, MoH did not separate previously infected, who are either unvaccinated or partially vaccinated, which could attenuate VE estimates.

12/n

While informative, this analysis is limited given lack of access to important confounders other than age; Israeli research groups w/ access to more complete data can do more rigorous analyses.

Article detailing all calculations & more conclusions:
13/n
covid-datascience.com/post/what-do-n…