Thread by @DiseaseEcology

New paper on biases in epi studies led by @AccorsiEmma
w/ @mlipsitch & many others.
Paper is extremely valuable in thinking carefully about how to interpret data. Sadly, *most* epi papers have failed to account for most of the biases they discuss.
S thread
link.springer.com/article/10.100…

Two big examples:
1) Efficacy of vaccination from observational studies
2) Studies of susceptibility & infectiousness based on secondary attack rate (SAR) data

1) Randomized control trials are the gold standard for assessing the efficacy of vaccines (& lots of other things, of course), because, theoretically*, people are randomized b/w vaccine & placebo groups.
Observation studies of vaccine efficacy (VE) aren't randomized, so,...

any factor that influences the likelihood of vaccination (e.g. access to health care) might also influence the likelihood of infection.
If these factors can't be measured & incorporated into analysis, then data can be badly biased.

*Aside: RCTs are great in theory, but clear differential side effects b/w vaccine & placebo can "un-blind" participants in RCTS & lead to bias if vaccinated engage in riskier behavior.

Simple example: 1st group to get vaccines were health care workers (HCW). Observational study of VE that matched HCW w/ non-HCW by age, gender, etc., would likely find that vaccines weren't that good at all, because HCWs have much higher COVID-19 exposure than gen pop.

This is silly example (no one would do this, right???), but any observational study is doing this with dozens of unknown variables, rather than something obvious like occupation. A nice recent thread illustrating when matching is & isn't working:

https://twitter.com/_MiguelHernan/status/1364700315044438023

2) Age & SARS-CoV-2 infectiousness & susceptibility
(Possibly the most contentious topic on COVID-19?)
Questions of whether schools should be closed to reduce transmission led to *many* studies trying to assess variation in infectiousness & susceptibility of kids vs adults.

The problem is that NONE (yes, 0) of the 100+ studies address all the biases laid out in this new paper. Key issues:
-Mis-ID index case
-Miss secondary cases
-Mis-interpret diffs in contact type/duration for diffs in susceptibility/infectiousness

Obvious Q is what to do with all the studies that have data that suffers from the biases described in this paper (for examples #1 & #2). Most obvious & necessary action is to acknowledge biases, qualify claims/conclusions. Sadly this is rarely done, except performatively.

Even this paper (??!!!) makes an unqualified claim that suffers from these biases: "evidence suggests
SUSCEPTIBILITY to infection increases somewhat with age [7]". The cited study suffers from all the same issues so carefully laid out here. Obviously we have a long way to go.

A second approach is to use other data to try to assess the impact of biases or confounders & adjust as needed. Same thread from above is nice example of this, and has an RCT to lean on!

https://twitter.com/_MiguelHernan/status/1364700315044438023

Similarly, if we had data from challenge (experimental infection) trials on age-infectiousness-susceptibility we could properly interpret SAR data. But that's unlikely to happen any time soon. Challenge trials are starting but only w/ 18-30 yrs (1daysooner.org)

In summary, paper is a fantastic read & we should try to remember each of these issues as we interpret new (& especially, observational) data.
Careful thinking, for which I'd argue @mlipsitch is the gold standard, is our best tool to understand the world.

Also, @mlipsitch has a partial thread on the paper here (that I hope he'll complete!)

https://twitter.com/mlipsitch/status/1365041619657981952