A short thread on vaccine efficacy versus vaccine effectiveness, covering:
- Idealized vs. real-world effects
- Direct vs. indirect effects
- Individual-level vs. population-level effects
- Phase III vs. Phase IV trials
👇👇👇
Phase 3 trials are conducted under idealized conditions. Everyone receives all doses on time, that have been properly stored, etc. The primary analysis is restricted, like to people without antibodies at baseline. We call the resulting estimate "vaccine efficacy." 2/8
Think "vaccine efficacy" as our best guess at the biological protection of the vaccine. When we talk about "vaccine effectiveness," this can refer to a few different things. One is "real-world effectiveness." If conditions are less than ideal, how well does the vaccine work? 3/8
Another concept of effectiveness is the "effectiveness of a vaccination program." Efficacy is an individual-level measure of protection, which we also call a "direct" effect. But vaccines can also have "indirect" protection, by preventing infection or reducing infectiousness. 4/8
We can measure the impact of a vaccination program at a population-level, instead of at an individual level. If you have an area where 50% of the population is vaccinated, versus an area where no one is vaccinated, how much better off is the partially vaccinated area? 5/8
This population-level effect is known as "overall vaccine effectiveness," and it captures both the direct protection and the indirect protection. In this way, it depends upon the vaccine coverage. It may also be highly location-specific. 6/8
A common post-licensure (Phase IV) design is a cluster randomized trial where the outcome is disease across both vaccinated and unvaccinated community members. These are known as "effectiveness trials."
Thus, while vaccine efficacy is narrowly defined, effectiveness is a broader term that captures both how well the vaccine works under real-world conditions, and, for overall effects, its population-level impact. 8/END bmcmedicine.biomedcentral.com/articles/10.11…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Astrazeneca/Oxford get a poor grade for transparency and rigor when it comes to the vaccine trial results they have reported. This is not like Pfizer or Moderna where we had the protocols in advance and a pre-specified primary analysis was reported. 1/5
AZN is evaluating their vaccine in multiple trials across the world, yet these are not embedded under a unified protocol. In fact, the trials seem to be quite different by country, in terms of populations, subgroups, etc. Based on the publicly available details I've seen. 2/5
With the exception of the US-based trial, I am not aware of details on how these trials are being monitored. Is there a centralized DSMB? Are they combining the accrued data? They seem to have combined events across Brazil and UK. Why not the other countries? 3/5
One question I had with the Pfizer results - is it possible that all cases were in younger (risk-taking) adults? Unlikely, but I couldn't tell. So it's great to see a breakdown of those 95 cases across different subpopulations. Representation is important! 6/10
Nice safety profile, and participants will continue to be followed. Remember - evaluation of vaccine safety never stops, even after vaccines are approved and deployed. We have an adverse event reporting system to continually monitor all vaccines. 7/10
Another key point - the Moderna vaccine has been formulated to not require the same intensive cold chain as Pfizer's vaccine (mRNA is unstable!). This has the potential to greatly simplify the logistics of rolling out the vaccine. 8/10
Of course, I'll start with the topline result of 94.5% efficacy (90 placebo cases, 5 vaccinated cases). For a high efficacy vaccine, 95 cases is a lot of data. In event-driven trials, the number of cases matters more than the number of participants. 2/10
Both Pfizer and Moderna's vaccines are mRNA-based. Thus, it's reasonable that results are similar. Still great to see! This is exciting because mRNA vaccines are designed for pandemics but were previously unproven. Adding a new tool to our toolbox! 3/10
Big news from Pfizer, with apparent high efficacy (>90%) based on 94 confirmed COVID-19 cases at their interim analysis.
A thread on how I interpret this news. Briefly:
"Celebrate, but let the process play out over time as intended." 1/8 pfizer.com/news/press-rel…
Vaccine trials are "event-driven." They continue until enough endpoints have accrued (here, lab-confirmed *symptomatic* infections). Statisticians can take planned "early looks" at the data, and so allow us to tell if a product is working exceptionally well (or not at all). 2/8
When the vaccine is highly effective, we need less data to see it. While trials are planned for 150+ total events, this is what we need for a 60% efficacy vaccine. I say this because 94 events is a lot of data for a vaccine trial, and even more so when efficacy exceeds 90%. 3/8
With @mlipsitch, our new piece in @ScienceMagazine on the open questions that may remain even after vaccine efficacy trials are completed. How well does the vaccine work in high-risk subgroups? And how well does the vaccine reduce onward transmission? 1/5 science.sciencemag.org/content/early/…
Vaccine efficacy trials are designed to measure individual-level protection, which serves as the basis for regulatory decision-making. But when we think about vaccination strategy at a population-level, other features like indirect protection come into play. 2/5
As a motivating example, to best protect high-risk groups like the elderly, we either need a vaccine that directly protects the elderly or we need a vaccine that can prevent younger people from transmitting to the elderly (ideally both!). 3/5
- How is vaccine efficacy calculated?
- Distinguishing between infection, disease, & severe disease.
- Measuring reduced infectiousness.
- Vaccine efficacy vs. effectiveness!
2) Vaccine efficacy (VE) measures the relative reduction in infection/disease for the vaccinated arm versus the unvaccinated arm. A perfect vaccine would eliminate risk entirely, so VE = 1 or 100%. This can be calculated from the risk ratio, incidence rate ratio, or hazard ratio.
3) Vaccine efficacy of 50% roughly means you have a 50% reduced risk of becoming sick compared to an otherwise similar unvaccinated person. Or you have a 50% chance of becoming sick given that you were exposed to enough infectious virus to make an unvaccinated person sick.