Thread by @jonlevyBU

1. Hi! Trained risk assessor here. Can we talk about this “1 in 5000” risk of getting #COVID19 if you are vaccinated, all of the things that are wrong with the number and how it is being used, and why I still think there is some value in the calculation? 🧵

2. First, for those unfamiliar, this first appeared (to my knowledge) in a column by @DLeonhardt in the @nytimes. He estimated a 1 in 5000 risk from stats in a few settings (UT, VA, WA). Let’s assume that his number is right. I still have a few major problems with it.

3. First, it is a daily risk, which is not how we commonly quantify risk. The column did say this directly, so it is not hidden from the reader, but many people dropped that nuance in talking about the number. Probably too much headline reading and not enough article reading.

4. Of course, this is not a one day pandemic, so a daily risk number isn’t terribly helpful. Translated to an annual risk, that’s around 7%.

5. People have a hard time gauging small numbers and small probabilities. I would guess that readers would be much more worried about a 7% annual risk than a 1/5000 daily risk, even if they are the same thing.

6. Second, it is not connected to time or place. Again, @DLeonhardt does say that 1/5000 is not the number everyplace and at all times, so it’s more about the application of this number and the fact that the headline lacked nuance.

7. The problem is, if a vaccine reduces the risk of a case by x%, then the absolute risk of a breakthrough case is higher when overall incidence is higher. Makes sense. But that means risks are higher now for vaccinated people than 3 months ago. Even worse for unvaxed, of course.

8. And case incidence varies wildly across the country. 2 orders of magnitude across counties. Layer onto that variable population density, different job activities, etc., and individual risks vary by many orders of magnitude.

9. This, I think, is the biggest problem - applying a population average risk to individuals, when the risk varies widely AND is influenced substantially by the individual.

10. If I work from home, wear an N95 in public, and live in a city with low case incidence, my risk is well below average. If I am an essential worker around unmasked people in a city with high case incidence, my risk is well above average. “1 in 5000” misleads in both cases.

11. Relatedly, getting COVID means very different things to different people, depending on individual vulnerability and the people around you. “1 in 5000” means something different if you live with unvaccinated or immunocompromised people.

12. All that said, let me praise one aspect of the “1 in 5000” calculation. Many scientific papers and other communications report relative risk reduction from vaccines, or the aggregate relative risk reduction from layered protection.

13. But that’s not often how people think. And when you talk about layered risk reduction, people can’t distinguish between 99% and 99.9% reductions even if they are extremely different from an absolute risk perspective.

14. So it’s good to put things in absolute terms. Not for silly comparisons with other risks, not leaving out time or place, and not ignoring the fact that my risk becomes yours. But so people can have a more realistic sense of what to do post-vaccination.

15. The risk to vaccinated people of getting COVID is not zero. That we can agree on. So these calculations can help us dispense with magical thinking and get people to take appropriate precautions. But we need to explain and use these numbers the right way.

16. 1 in 5000 is not a number to be dismissed, especially on a daily basis. It is a number to be better understood at an individual level, with individual and collective actions to reduce risk for the vaccinated and unvaccinated alike.

17. Lots of comments on whether 1 in 5000 daily risk for vaccinated is reasonable and plausible. I didn’t calculate it - but we can do a quick back of the envelope calculation to see if it passes the smell test. 🧵part 2…

18. Let’s say there are around 150,000 new cases per day and 54% of the country is vaccinated (got that from nytimes.com/interactive/20…). Call the US population 330 million.

19. And let’s assume the vaccinated are 5 times less likely to get infected than the unvaccinated, per the CDC.

20. Then it’s just algebra.

Let X be the rate in the vaccinated, and 5X the rate in the unvaccinated.

Overall daily rate is 150,000 divided by 330 million, or about 1 in 2200.

21. So it is .54X + .46(5X) = 1 in 2200

(Parenthetically, doing math on your phone while your 7 year old is throwing paper airplanes at you creates space for errors. Apologies in advance if I botch something…)

22. Solving for X, it is around 1 in 6000.

In other words, very roughly, the daily risk is around 1 in 6000 for the vaccinated, 5 times higher for the unvaccinated.

Very rough calculation, not controlling for anything, but it shows that the @DLeonhardt number is roughly right.