Thread by @JamesWard73

There been something bugging me for a while about how my model (and others) works. And I think I’ve finally pinned it down. It’s subtle and technical, but I think might turn out to be surprisingly important. And it’s all to do with the “leakiness” of the immunity model. 🧵

The difference between a “leaky” vaccine and an “all-or-nothing” vaccine has been well-covered by others – my favourite explanation is in this thread:

https://twitter.com/erlichya/status/1419219698714042371?s=20

As it happens I’m not particularly happy with the “leaky” terminology – it could mean a number of different things, and so could be misinterpreted. I prefer to think about this in terms of the variability of the vaccine effectiveness (VE) at individual level.

At one extreme (the “leaky” end), the vaccine response for every individual is the same. So if the overall VE is 70%, then everyone gets 70% protection. (meaning that when they are exposed, they are 70% less likely than an unvaccinated person to get infected).

At the other extreme (“all or nothing”), the vaccine response is as varied as it can be while still preserving the same average VE of 70%. So in this model, 70% of the people get 100% protection, and the other 30% get nothing at all.

Of course, the reality is probably at neither of those two extremes; there will be some variability in the vaccine response, but not as far as the all-or-nothing model implies. Instead, it will follow some kind of distribution around the average 70% - here’s one I made up:

OK, but why does any of this matter? Well, really for two reasons. Firstly, in a leaky/homogeneous response model, the VE will appear to wane over time *even if there is no actual waning going on* - as explained by @erlichya:

https://twitter.com/erlichya/status/1419219712253247489?s=20

Having played around with this, using plausible parameters, it’s quite possible to find an apparent waning effect that is 2-3% per month higher than the actual waning of VE (note the distortion increases with prevalence, and gets bigger over time as VE reduces).

So what this means is that when we’re observing waning at say 7-10% per month, it’s possible that a fair chunk of that is a statistical artefact caused by the leakiness of the vaccine. Which would be good news, because it means the real waning rate is slower.

BUT there’s a second reason why leakiness is important, which is not such good news, and has been less widely discussed (or if it has, I’ve missed it). To explain this, we need to consider what happens when someone who has been vaccinated gets infected.

In an all-or-nothing model, we can be sure this is someone who had no immunity, because the people with 100% protection can’t get infected, so it must be one of those who had 0% protection. So that individual is going to move from having 0% protection, to a much higher level...

– maybe 80% protection, post-infection. (in fact, in an all-or-nothing model, it will be an 80% chance of having 100% protection, and 20% chance of having none, but that doesn’t really matter here). So in that case, there is a gain of 0.8 units of immunity in the population.

Now consider the “leaky” model. Here everyone post-vaccination has the same, partial, protection at 70%. So if someone gets infected they were already starting on 70% protection. After infection, they will have higher-grade immunity – let’s say they move up to 90% protected.

But now the gain in immunity is only 0.2 units, rather than 0.8 units. This is important, and worrying, because of what it means for the endemic equilibrium. Remember, at equilibrium we need the gain in immunity from infection + vax to balance the loss of immunity from waning.

So for the same level of waning, we’re going to need a lot more infections at the equilibrium in a “leaky” model than we do in an “all or nothing” model. It’s not quite as bad as it might seem from the above, because waning effects mean the starting VE won’t be as high as 70%.

To get a better estimate of the effect, I tried this out in my “toy” endemic model – the one I used for this thread:

https://twitter.com/JamesWard73/status/1444234690597113859?s=20

. That model used an all-or-nothing approach, and here it is with around 5% per month waning, and a dose of seasonality:

Now if I switch the model to using “leaky” immunity (not quite as easy as that makes it sound, but you’re not here to hear about my modelling headaches), I get quite a different result, with an equilibrium level *3 times higher* than before:

So this nice little interesting technical detail in the modelling turns out to mean that maybe 3 times more people could be catching covid per year in the endemic endstate, then? Well, maybe. As we noted above, the truth is probably somewhere between the two extremes.

And as we saw, leakiness also means that our observed estimates of waning may be over-stated. So these two points may offset, to a degree. If we have high leakiness, the direct effect is to increase the equilibrium level, but if it also means the real level of waning is lower,

...that would reduce the equilibrium rate of infection. Conversely, if we have high variability of individual response, that should mean a lower equilibrium rate, but it also means that the observed high rates of waning are more likely to be correct.

Still, it feels quite important to work out where we sit on this spectrum between “leaky” and “all or nothing”. And it doesn’t feel impossible to do that: we already know some things about the variability of immune response (antibodies, T-cells etc.) at an individual level.

And we’re also starting to build a view of the “correlates of protection” i.e. the conversion between immune response, and observed protection from the virus. So maybe we could estimate the distribution of VE at an individual level?

In fact, maybe this has already been done? (in which case I’d welcome any pointers to the relevant analysis / datasets). If not, it feels like this would be a helpful exercise, but I think needs someone with a bit more immunological expertise than me, to avoid messing it up.

Once that was done, we’d need to do some additional modelling to work out what that distribution would mean for observed rates of waning, and for the equilibrium rate of infection – but that’s not particularly difficult, and I’d be happy to do that part of the work.

I’d also be interested to know whether the Warwick / Imperial / LSHTM models use an all-or-nothing, “leaky” or in-between model for immunity? @GrahamMedley @DrLouiseDyson @EdMHill @mjtildesley @BarnardResearch @_nickdavies @lilithwhittles @erikmvolz any info appreciated. /end