Thread by @michaelzlin

We usually hear only overall top-line efficacy figures of 66% against symptomatic disease and 93% against hospitalization. We also hear that in the J&J trial, there were no deaths in the vaccinated group. That's across the entire tested population. But even then, to be accurate

... there is considerable imprecision in our knowledge of how well J&J protects against more severe outcomes. The 93% protection v hospitalization has 95% confidence intervals of 73-99%. This large range is due to the small # of people actually hospitalized during the trial.

Our knowledge of protection from death is even weaker. The fact 0 vaccine vs 6 placebo recipients died is often touted as 100% protection against death, even by public health officials. This is unlikely to be the case, and scientifically inaccurate.

The 95% confidence interval for protection against death was never reported but will be ≤0 to ≥100%. (Yes you can have intervals outside 0-100.) How do we know that? Because the CI was calculated for severe disease outcomes, which includes death.

There were 1 vaccine and 7 placebo severe outcomes in the US. This produced a nominal protection level of 86% but with a 95% confidence interval of -9% to 100%! That is, there is a 95% chance that the true protection value is between -9% and 100%. That's not very certain.

The sample size for evaluating deaths is even smaller (0 vs 6, compared to 1 vs 7), so we can deduce the CI will be larger; it will probably cover the entire 0 to 100 range.

That's so we're all clear on what the numbers actually say. No % protection v death can be calculated.

However, we can assume there's good protection against death, because there's this 73-99% protection against going to the hospital. As long as J&J recipients are not more likely to die than placebo recipients once in the hospital, then that % protection should hold for death too.

Now with 10 million Americans having been vaccinated with J&J, you'd think we'd have reports on real-world efficacy against disease, hospitalizations, and deaths. Unfortunately, and surprisingly, I could only find one study, and it's uninformative
medrxiv.org/content/10.110…

The above study reports 77% efficacy against disease, verifying the trial results, but no effect on hospitalization or death could be observed due to small numbers in the small numbers of subjects over the very short trial period.

Yet with 10 million J&J vaccine recipients, the vast majority now vaccinated for more than 2 months, CDC should have the data to accurately assess protection against disease, hospitalization, and death. With Delta on the rise, it would be useful to know!

So without exact numbers, the best we can do is to use the 93% protection against hospitalization (95% CI 73%-99%) as an estimate against death, in the overall population. That's not bad. And as discussed before, J&J holds up well to past variants, so will likely hold up to Delta

So if I were a young and healthy J&J recipient, I wouldn't be too concerned. However vaccine efficacy does seem to be affected by preexisting conditions. Here's a breakdown by age and comorbidities (obesity, diabetes, cardiovascular disease). Note both the estimate and the CIs.

The 95% CIs all overlap, so that means we can't conclude any differences between the groups with 95% statistical confidence. However there is a trend where ≥60yo with comorbidities may do worse than ≥60yo without.

If you break this down by comorbidity, it seems the difference can be assigned to hypertension, diabetes rather than obesity (among the 3 most common conditions). Again all differences are non-significant statistically, so they are hypotheses for the moment, not conclusions.

Now those numbers are about protection against symptomatic disease, not hospitalization or death, but if it looks like protection against symptoms is dropping in ≥60yo with comorbidities, there's likely going to be some drop in efficacy against hospitalization/deaths too.

Even if the drop in efficacy against hospitalization/deaths is small, recall that being ≥60yo with comorbidities already markedly increases risk of death. Thus that group will want the highest possible protection *even if the vaccine worked as well for them as for other groups*.

So given these data, if I were ≥60yo and diabetic and vaccinated with J&J, would I be asking my doctor if he/she thinks it's okay for me to get a RNA shot as a booster now, so I can be prepared before Delta creates an epidemic in my neighborhood? Yes I would.

In case you are wondering where to find such data presented clearly in one place, it is the WHO background document. They have these for each vaccine they have considered for approval.

who.int/publications/i…

BTW lower protection for ≥60yo with comorbidities is also seen for Pfizer in this Israel study (Israel, a much smaller data, seems to be able to actually gather and analyze data despite having much less of it). Here just look at red (29-50 days post-vax).
medrxiv.org/content/10.110…

This is because vaccine efficacy is defined as relative to placebo for whatever group is being studied. The vax being 60% effective for ≥60yo with comorbidities means vaxxed people in that group are protected 60% vs unvaxxed. But the outcomes are worse than other groups.

Finally, you will notice that the data suggesting ≥60yo with comorbidities might be interested in getting a RNA booster are all old. So why are we only thinking about this now? I think it's 3 reasons:

• initially we didn't have enough RNA vax for everyone, so we didn't have the luxury of giving RNA boosters to J&J recipients.
• the data on effects of age and comorbidities was not noticed since media attention was on the top-line number

• earlier we thought vaccination would give us herd immunity but with incomplete vaccination and now Delta causing worse outcomes and being more contagious, we need to further improve our protection levels among the vulnerable vaccinated