Thread by @mlipsitch

@CT_Bergstrom @jsm2334 Certainly this kind of bias merits consideration. I think the particular figure cited in that table is an example of Simpson's paradox, which is a special type of confounding.

@CT_Bergstrom @jsm2334 For those new to these terms, confounding is just the problem that (in this case) vaccine is not randomly distributed in the population, so the vaccinated have different risks from the unvaccinated for reasons other than their vaccine: in this case, age.

@CT_Bergstrom @jsm2334 Simpson's paradox is an extreme form of confounding in which a combined analysis for two groups of people gives an unusually misleading estimate, relative to the (more) accurate estimate for each group individually.

@CT_Bergstrom @jsm2334 As en.wikipedia.org/wiki/Simpson%2… exemplifies, defining Simpson's paradox precisely is hard, so I'd prefer to just call it confounding, which, again, it is, though it can only happen under certain circumstances

@CT_Bergstrom @jsm2334 For a clear and precise discussion of the relationship see academic.oup.com/ije/article/40… by the ever clear-headed @_MiguelHernan @CAUSALab

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab I don't think that this is the explanation for the most striking result, on slide 7 of this deck from @IsraelMOH which I shared recently as it is hard to find online gov.il/BlobFolder/rep…

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH that result is that Jan vaccinees, over 65, for severe disease, have a VE of 55% or so, after adjusting for two key confounders: week and 10-year age group (up to 85+), as well as sex (not much of a confounder)

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH So to invoke confounding by age, you'd have to assume that the residual confounding within these age groups is very large. Similar result for Feb (60% point estimate) and higher VE for the March vaccinees in that age group (who are probably quite unusual given the rollout speed

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH This is consistent with what you'd expect if waning of immunity to, combined with Delta, is causing a drop in VE to severe disease.

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH This drop was not seen as far as I know in the data on infections pre-Delta (up to mid-June), certainly not up to early April in a very similar analysis by the same group of people at MOH thelancet.com/journals/lance…

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH If indeed this is a new finding (ie only in the late June and after data) it also doesn't make much sense to me to think this is due to depletion of susceptibles bias (which is one of my favorite candidates and one I've written a lot about) ....

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH because the drop came during a lull in incidence, while depletion of susceptibles requires incidence medrxiv.org/content/10.110…

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH (more general discussion here cambridge.org/core/journals/…)

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH So I'm all in favor of discussing potential biases that could create the observations we've seen, but I would say in this case I don't think that confounding or depletion of susceptibles bias is a full explanation for the observed decline which is concentrated in the

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH earliest-vaccinated, oldest group in the Delta era

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH I'll also mention that along with many in Israel I was uncertain about the early indications of waning immunity to infection because of possible differential testing, but I don't think that could explain observed reduced VE for severe disease, a well-ascertained outcome

@CT_Bergstrom @jsm2334 @_MiguelHernan @CAUSALab @IsraelMOH Key thing is for the analyses to be public so we can discuss possible explanations for the observed data and ways to distinguish among them. Hopefully will have a robust discussion and pressure-test these findings; current views, could change in light of new evidence or analysis