This simple statement could be the central dogma of #flu #epistemology. @mtosterholm likes to say "If you've seen one flu pandemic, you've seen one flu pandemic." May apply to virtually all of biology, which relies heavily on induction about a combinatorial subject -- impossible?
What is interesting is the extent to which, although aware of the hazards of generalizing from (usually small) past samples, scientists often can't help themselves. I'll start with a mea culpa: Before 2014, if someone had told me they were developing Ebola vaccines for Africa,
I would have thought either that they were lying and actually doing it for biodefense, or that they were unwisely pursuing something unnecessary, because time after time (25x to be exact) Ebola epidemics had been contained w simple nonvaccine interventions.
I'd have been wrong. In simple terms, doing induction on biology is risky, especially when N is small. Quantifying that, jamanetwork.com/journals/jama/… reminds that if something hasn't happened after N years, we can't conclude it won't happen, only that the probability per year is...
Probably less than 3/N (Frequentists and p-value haters and continuity correction lovers please don't hassle me about this heuristic statement). For example if it hasn't happened in 15 years, we can be reasonably confident the chance of its happening is less than ~ 1 per 5 years.
That assume the risk is constant through time. If we think the risk is changing through time (or if we are generalizing across biological systems and systems have different risks), things get even less certain.
Yet at least in infectious diseases people with doctoral degrees say with some frequency things that ignore this humbling fact about induction. As noted I have (or would have if asked at the time) committed this sin. Other claims I've heard in my research area:
Serotype replacement didn't happen with the H. influenzae b vaccine, so it won't happen with S. pneumoniae (Spoiler: it did happen, to a dramatic degree in some places, though net effects of Spn vaccine have been highly beneficial).
(I wrote at the time about why that particular leap didn't make even theoretical sense due to differences in the ecology of the organisms ncbi.nlm.nih.gov/pmc/articles/P…)
Another such argument: If H5N1 was going to go pandemic, it would have done so already. None of the wise people quoted in @HelenBranswell's story say that, but it is something that is talked about a lot informally and I think believed by many in the flu community.
Those who know me will suspect that I am getting to a connection to the recently renewed #gainoffunction studies where avian flu viruses are altered to become more mammalian-transmissible. sciencemag.org/news/2019/02/e…
You're right. The need for humility on induction about what can't happen is relevant to both the risk and the benefit side of these studies. On the risk side, proponents noted repeatedly during the debate that no infection had ever escaped a lab in one of these experiments
But the sample size was tiny, probably less than 10 lab-years of experiments -- and no one would say that "Oh I'm pretty sure the risk is less than 1 in 3 lab-years of creating a lab-acquired infection with a potential pandemic pathogen" is reassuring.
On the benefits side: Proponents of creating potential pandemic pathogen #influenza strains in the lab argue that this will facilitate identification of high- and low-risk strains in nature for prioritizing response efforts (eg culling poultry with high-risk strains)
This argument has several flaws. 1. the work done to date and published in Science, Nature and Cell has identified nothing we didn't already know about in terms of risky changes in the virus (table in academic.oup.com/jid/article/21… .
2. More directly related to the problem of induction: Epistatic interactions (genetic background changing the effect of a mutation) mean that what we learn in one strain is at best imperfectly predictive of what is dangerous in natural strains (fig in cidrap.umn.edu/news-perspecti…)
3. Each time we try to develop "rules" for what is needed to cause a pandemic, we are proven wrong. For example the 2009 pandemic violated the top 4 rules: must be a new subtype (H1N1 was already around), must have strongly human-adapted HA tropism (early pdm strains didn't)...
must have PB2 E627K (while she was working on CRISPR, on the side @doudna_lab showed why this wasn't necessary pnas.org/cgi/pmidlookup…, and must have pH of fusion in a particular range (early isolates were outside range). We review evolving understandings elifesciences.org/articles/18491
The point: while studies of one or a few strains can provide existence proofs, they cannot plausibly show that something in flu is impossible. Thus they can at best contribute incrementally to pandemic risk prediction, not lead to major breakthroughs in that area,...
which will remain the realm of opinion and educated guessing for the coming centuries, because with N=4 known pandemic emergence events (or even 10x that many) we cannot do meaningful inductive guessing, much less rigorous inductive reasoning.
\end{tweetstorm}
In summary, we all want our science & experience to be generalizable. Human nature! In infectious diseases at least, very rarely are we justified in making strong predictions esp. about what CAN't happen. Humility is a virtue esp when considering dangerous science experiments.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Marc Lipsitch

Marc Lipsitch Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @mlipsitch

2 Aug
usually agree with both @StevenSalzberg1 and @nataliexdean. In this case agree with the cautious view. Even if safety were known (which I don’t think it is for this) RCT r really important for efficacy. Alternatives, which both @nataliexdean & I work on, are full of pitfalls.
The article by @StevenSalzberg1 just gets some things wrong. Published test of eg the Oxford vaccine had 126 and 253 vaccinated, enough to detect adverse events if occur in 2.3% and 1.4% respectively.
Read 7 tweets
3 Jul
@DiseaseEcology @US_FDA @rebeccajk13 @Steve_Bellan I may not be explaining well, and it's a subtle thing I'm trying to say. I think it is important to know effect on serologic infection. My point is that there may well be a vaccine that 1) makes disease less likely/severe, 2) makes shedding much less and 3) permits seroconversion
@DiseaseEcology @US_FDA @rebeccajk13 @Steve_Bellan Such a vax would look good on the disease endpoint (esp if they use our approach or similar to correct for missed infections), but null on the infection endpoint. This vaccine would be very good for herd immunity (and direct protection) but the analysis might miss that fact.
@DiseaseEcology @US_FDA @rebeccajk13 @Steve_Bellan Basic issue I think is that infection and infectiousness => seroconversion but seroconversion !=>infectiousness
Read 5 tweets
2 Jul
This week @US_FDA called for confirmed infection (+-symptoms) as a primary or secondary endpoint in #SARSCoV2 #vaccine trials. Last year @rebeccajk13 @Steve_Bellan Rui Wang and Matt HItchings we recently proposed efficient, unbiased approach: academic.oup.com/aje/article/18…
Link to the FDA guidance fda.gov/regulatory-inf…
One subtle point: preventing infection detected serologically (without symptoms) may be a very difficult endpoint for many vaccines.
Read 6 tweets
23 Jun
I agree with a lot of this thread. Primarily - in any time, but especially these days, it is inspiring to see a government agency release data, get bad press, and respond by releasing more data and asking for expert input.
Also impressive was the evidently hard-working, experienced, resourceful team of people trying to make this program a success, and the consistent effort to confront unsatisfactory numbers with feasible means for improvement. Given mission to make TTI successful, very impressive
But I think behind the NYT story and behind other critiques of tracing for example nationalgeographic.com/science/2020/0…
Read 18 tweets
19 Jun
On its face (and maybe even after some careful consideration), this article is concerning. The First Covid Vaccines May Not Prevent Covid Infection a.msn.com/00/en-us/BB15t…
It points out as as been noted before blogs.sciencemag.org/pipeline/archi… that the Oxford vaccine in macaque studies does not prevent (or even visibly reduce) the nasal shedding of virus RNA -- infection and perhaps contagiousness are the same, but vaccinated macaques don't get sick
and have much less virus RNA in their lungs
(from biorxiv.org/content/10.110…) -- concern originated with Bill Haseltine forbes.com/sites/williamh…
Read 18 tweets
7 Jun
My conclusion is that we should be trying to work together to fight injustice, reduce transmission, and where possible resume some normal activities, including protests.
Any effort to assign numbers to one activity is full of assumptions, including the assumption that R stays constant over time -- something we can change if we choose by making greater efforts to suppress transmission.
I offer the comparison simply to point out the risk of trying to assign responsibility to one decision in the face of all the dependencies -- consequences of infectious disease cases are dependent on the transmission environment, which is in our control.
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!