Thread by @Jeffely

The base-rate fallacy is about to become a daily nuisance when it comes to processing outbreak data in highly vaccinated societies. Here's a cautionary tale.

Today the Wall Street Journal has the following alarming news. In the recent surge of cases in Israel involving the Delta variant, half of all newly infected adults had been fully vaccinated. I almost cried when I read that.

wsj.com/articles/vacci…

But hang on. Isn't almost everyone fully vaccinated in Israel? Thought experiment. Suppose I told you there was a superspreader party attended by 100 people and half of those infected had been fully vaccinated. Does that make you worry about vaccine effectiveness?

What if I then told you that in fact 99 of the 100 people at that party had been fully vaccinated. Well these facts together tell you that 1 person was unvaccinated and got infected and 1 of the 99 vaccinated people got infected. In other words the vaccine was 99% effective.

In general if you know that a fraction r of a population is vaccinated and from that population equal numbers of vaccinated and unvaccinated people have gotten infected from some event, then (1-r)/r is the relative probability of infection vaccinated versus unvaccinated.

Why is that? Let q be the fraction of individuals exposed to the virus by the event, and let p_v and p_u be the probability of becoming infected conditional on exposure for vaccinated and unvaccinated respectively. Then we have

q*p_v*r = q*p_u*(1-r)

i.e.
p_v/p_u = (1-r)/r

Back to Israel: my back of the envelope calculation is that about 83% of adults are fully vaccinated.

(I can't find the exact percentage but I know from the NYT that ~60% of the overall population is fully vaccinated, I "know" from Wikipedia that adults make up ~72% of the Israeli population and I recall that vaccination of children in Israel has barely begun.

This gives an estimate of about 83% of adults fully vaccinated.)

So taking r = .83, the ratio (1-r)/r comes to about .2

This translates to a vaccine effectiveness of 80%.

Remember that my 83% was just an estimate. But more importantly, another detail from this outbreak in Israel is that it started with some large clusters in schools. Certainly the fraction of adults in a school context who are vaccinated is much larger than the baseline.

We could make more guesses but instead lets reverse engineer it. What value of r would give us back the effectiveness estimates of 90% that we saw in the early vaccine trials (before Delta)?

(1-r)/r = .1

yields r = .91

So as long as ~91% of the adults exposed in the early days of the outbreak in Israel were fully vaccinated, this alarming news is actually perfectly consistent with the Pfizer vaccine being no less effective against Delta than it was against old school covid.

In fact it would be a *routine occurrence* that fully vaccinated individuals make up half of new infections. Indeed there will shortly come a day when *the vast majority* of new infections will be fully vaccinated people.

And that won't make us worry one bit. In fact we should be cheering once that starts happening. N

PS: None of this is proof that Delta is nothing to be worried about. I am certainly worried about it. Its early days. But sometimes statistics can look more scary than they really are.