Thread by @JamesWard73

@kallmemeg has kindly nominated me to undertake some vaccine efficacy (VE) estimates from the data in Table 6 of the latest PHE Tech Briefing. I know the number of deaths in the “double vaxxed” column has been causing concern, but I think it’s OK. 1/n

https://twitter.com/kallmemeg/status/1403290578704547844?s=20

My overall conclusions are:
1)The vaccine’s efficacy vs. disease is hard to deduce from this table, as it’s sensitive to the exposure risk assumption, but a rough estimate is consistent with the PHE’s figures in Table 18 (i.e. 33% after 1 dose and 80% after 2 doses) 2/n

2)The VE vs. hospitalisation and death appears robust to different assumptions of exposure risk, and implies that the vaccines are maintaining good protection (~80%) after 1 dose, and very strong protection {>95%) after 2 doses. 3/n

I also know that some have expressed concerns about the accuracy of these calculations in previous weeks, so I want to be clear: it’s a back-of-the-envelope estimate and is subject to distortion. But I’m confident it’s in the right ballpark, otherwise I wouldn’t share it. 4/n

This week, I’m also going to give you a sensitivity analysis which attempts to correct for one of the main possible distortions, which will show you which figures are unstable (and therefore maybe not to be trusted) and which ones are more robust. Details in a bit. 5/n

So, to start with I have repeated the calculations from previous weeks, using the new data from Table 6. The key to interpreting this is to understand that the relative risk in the double-vaxxed, single-vaxxed and unvaxxed groups isn’t even; for example the double-vaxxed are 6/n

…overwhelmingly those in older age groups (as well as the clinically vulnerable, and health/care workers) and therefore have a much higher level of risk of hospitalisation and death from covid than those in the other groups. 7/n

To adjust for this, I’m working out the rough age-group composition of each vaccine status group, and mapping this against my estimated (pre-vaccine) levels of death and hospitalisation risk, to work out how much of the risk sits in each vaccine category. 8/n

In doing this, I’m also making an adjustment for the fact that people in younger age groups tend to be more sociable, and are therefore more likely to be exposed, and infected. This cuts slightly the other way, moving risk to the unvaxxed groups. 9/n

Combining these effects, I can then say that while the double-vaxxed would account for about a third of the exposure risk in normal circumstances, they would also account for 75-80% of the hospitalisation risk. So if we then observe that in reality they only account for… 10/n

…6% of cases, and 9% of hospital admissions (as Table 6 informs us), then that tells us something quite powerful about the impact of the vaccines. And even though the number of double-vaxxed deaths looks large, at 29% of the total, we’d normally expect that group… 11/n

…to account for about 85% of the deaths – or about 14 times the number of deaths in the unvaxxed group. So the fact that in reality it’s 29% of the total, or more relevantly about *half* the number of deaths in the unvaxxed group, tells you that the vaccines are working. 12/n

So my revised estimates by this method are as follows:
-VE vs being in the case data: ~50% after 1 dose and ~90% after 2 doses (I’ll explain in a second why those might be a bit too high)
-VE vs. hospitalisation and death: ~80% after 1 dose and >95% after 2 doses. 13/n

Now, as I flagged above, this is a rough estimate and subject to potential distortion. The most obvious issue is that the distribution of exposure is not the same now as earlier in the epidemic, and in particular Delta is circulating most actively in younger age groups. 14/n

This means that the chances of someone in the older, double-vaxxed group being exposed to the virus is much less than normal, and so part of the risk reduction we’re seeing is from that reduced exposure, not from the effect of the vaccine. So our VE will be over-estimated. 15/n

Now, you could argue (with some force, I think) that the *reason* Delta is circulating in the younger groups and not the older is very much due to vaccines, and so it wouldn’t be wrong to include this effect in our VE estimate – in effect, you’d be measuring a composite of 16/n

…the individual protection effect of the vaccine (our traditional VE measure), and the ‘herd protection’ effect that arises from the older groups being heavily vaxxed, and the fact that they tend to socialise more with people their own age, and so will protect each other. 17/n

But let’s take the more pure and correct view, that we want our VE to measure only the individual protection effect. So I need to correct the exposure risk, and I can do this by working out the case rates in different age groups over the last few weeks, and making some… 18/n

…assumptions about the contact rates between different groups. Applying those adjustments, I get new estimates as follows:
-VE vs being a case: ~35% after 1 dose and ~80% after 2 doses
-VE vs. hospitalisation and death: ~80% after 1 dose and >95% after 2 doses. 19/n

Two things to note: Firstly, those case VE estimates are remarkably similar to PHE’s estimates for VE vs. symptomatic disease for Delta, as shown in Table 18 of the Tech Briefing: (and you’re going to have to trust me that I didn’t fiddle this, it just came out that way) 20/n

Secondly, the estimates for VE vs. severe outcomes (hospitalisation and death) hardly shifted. There were some minor movements but only by ~2% after 1 dose, and <1% after 2 doses. So I’m pretty confident that those numbers are robust to different exposure assumptions. 21/n

This is not the only source of potential distortion; for example if Delta shifted the risk of hospitalisation between different groups, I won’t be picking that up, and it could bias the calculations. But I think that’s unlikely to shift the VE numbers very significantly. /end