A few tweets on a topic that keeps coming up in discussion. There are many different types of vaccine efficacy - efficacy against infection, against transmission, against disease, and against severe disease - and these can vary for a single vaccine. How are they related? 1/5
Efficacy against infection will by necessity be lowest, because if a vaccine protects you from infection, it also protects you from transmitting to others and getting symptoms. We have a little data on this from Moderna and Oxford, but will get more from antibody testing. 2/5
Even if a vaccine does not prevent infection, it could make you less infectious by reducing viral load, reducing duration of infectiousness, or by preventing symptoms like coughing/sneezing. This effect is hard to measure without contact tracing or cluster randomized studies. 3/5
Our group's household secondary attack rate meta-analysis has gained traction, but not for the reasons I'd hoped for. We did not conclude "no asymptomatic or pre-symptomatic spread" of SARS-CoV-2. A short explanation of what we did observe. 1/7 jamanetwork.com/journals/jaman…
Using only the household studies included in our main analysis, we conducted a sub-analysis breaking out index cases designated as symptomatic versus asymptomatic/pre-symptomatic. We observe lower transmission from this latter group, though there was much less data. 2/7
Since we are relying upon other studies in the literature, we were unable to separate out fully asymptomatic index cases (never develop symptoms) from pre-symptomatic index cases. But others have tackled this problem directly. Their conclusions below. 3/7 medrxiv.org/content/10.110…
As @hankgreen nicely points out, we have to be careful that "we don't know whether the vaccine reduces transmission" doesn't morph into "the vaccine doesn't reduce transmission." How do we communicate this uncertainty? A few thoughts. 1/7
First, vaccine efficacy against infection can't be higher than vaccine efficacy against disease. If something prevents infection, it also prevents disease. But vaccines can work by preventing symptoms, and so give an extra boost to efficacy against disease. 2/7
So while we expect vaccine efficacy against infection to be lower, we aren't sure how much. We have a bit of data from the UK/Oxford and Moderna trials showing reduced infection, but we are waiting on antibody testing data from these and other trials. 3/7
Out in @TheLancet, results from the Oxford/AZN trials, including more detail on the low dose results. Notably, the low dose recipients "received their second dose after a substantial gap." Only 0.8% received a second dose within 8 weeks of the first. 1/5 thelancet.com/lancet/article…
Recall that the low dose results were only from adults 18-55, only during a short time window, and only in the UK. Per reviewer request, they restricted the standard dose analysis to a similar group. We still see separation (middle rows), but with more uncertainty. 2/5
Overall, the 62% result for the standard dose regimen appears robust, and meets pre-specified criteria (>50%). But I am still not sure what to make of the low dose result. Is it the longer gap between doses? The low dose? Both? And there remains no data for older adults. 3/5
With respect to the AstraZeneca vaccine, I am guessing people think my objection is to science by press release, and that I want a peer-reviewed publication. But no, not really. What I want is reliable and definitive evidence to inform policies impacting millions. 1/4
If the answer is that AstraZeneca needs to go back and add a new half-dose arm to their trials so that they can prospectively evaluate its efficacy in diverse subgroups, then we have to carefully consider the value of a peer-reviewed publication at this moment. 2/4
We’ve written about this in @NEJM. Basically, there are risks to publishing results that are “promising but inconclusive.” Though it seems slower at the time, in the long run it is better to generate the conclusive evidence while we still can. 3/4 nejm.org/doi/full/10.10…
Astrazeneca/Oxford get a poor grade for transparency and rigor when it comes to the vaccine trial results they have reported. This is not like Pfizer or Moderna where we had the protocols in advance and a pre-specified primary analysis was reported. 1/5
AZN is evaluating their vaccine in multiple trials across the world, yet these are not embedded under a unified protocol. In fact, the trials seem to be quite different by country, in terms of populations, subgroups, etc. Based on the publicly available details I've seen. 2/5
With the exception of the US-based trial, I am not aware of details on how these trials are being monitored. Is there a centralized DSMB? Are they combining the accrued data? They seem to have combined events across Brazil and UK. Why not the other countries? 3/5
A short thread on vaccine efficacy versus vaccine effectiveness, covering:
- Idealized vs. real-world effects
- Direct vs. indirect effects
- Individual-level vs. population-level effects
- Phase III vs. Phase IV trials
Phase 3 trials are conducted under idealized conditions. Everyone receives all doses on time, that have been properly stored, etc. The primary analysis is restricted, like to people without antibodies at baseline. We call the resulting estimate "vaccine efficacy." 2/8
Think "vaccine efficacy" as our best guess at the biological protection of the vaccine. When we talk about "vaccine effectiveness," this can refer to a few different things. One is "real-world effectiveness." If conditions are less than ideal, how well does the vaccine work? 3/8
One question I had with the Pfizer results - is it possible that all cases were in younger (risk-taking) adults? Unlikely, but I couldn't tell. So it's great to see a breakdown of those 95 cases across different subpopulations. Representation is important! 6/10
Nice safety profile, and participants will continue to be followed. Remember - evaluation of vaccine safety never stops, even after vaccines are approved and deployed. We have an adverse event reporting system to continually monitor all vaccines. 7/10
Another key point - the Moderna vaccine has been formulated to not require the same intensive cold chain as Pfizer's vaccine (mRNA is unstable!). This has the potential to greatly simplify the logistics of rolling out the vaccine. 8/10
Of course, I'll start with the topline result of 94.5% efficacy (90 placebo cases, 5 vaccinated cases). For a high efficacy vaccine, 95 cases is a lot of data. In event-driven trials, the number of cases matters more than the number of participants. 2/10
Both Pfizer and Moderna's vaccines are mRNA-based. Thus, it's reasonable that results are similar. Still great to see! This is exciting because mRNA vaccines are designed for pandemics but were previously unproven. Adding a new tool to our toolbox! 3/10
Big news from Pfizer, with apparent high efficacy (>90%) based on 94 confirmed COVID-19 cases at their interim analysis.
A thread on how I interpret this news. Briefly:
"Celebrate, but let the process play out over time as intended." 1/8 pfizer.com/news/press-rel…
Vaccine trials are "event-driven." They continue until enough endpoints have accrued (here, lab-confirmed *symptomatic* infections). Statisticians can take planned "early looks" at the data, and so allow us to tell if a product is working exceptionally well (or not at all). 2/8
When the vaccine is highly effective, we need less data to see it. While trials are planned for 150+ total events, this is what we need for a 60% efficacy vaccine. I say this because 94 events is a lot of data for a vaccine trial, and even more so when efficacy exceeds 90%. 3/8
Vaccine efficacy trials are designed to measure individual-level protection, which serves as the basis for regulatory decision-making. But when we think about vaccination strategy at a population-level, other features like indirect protection come into play. 2/5
As a motivating example, to best protect high-risk groups like the elderly, we either need a vaccine that directly protects the elderly or we need a vaccine that can prevent younger people from transmitting to the elderly (ideally both!). 3/5
- How is vaccine efficacy calculated?
- Distinguishing between infection, disease, & severe disease.
- Measuring reduced infectiousness.
- Vaccine efficacy vs. effectiveness!
2) Vaccine efficacy (VE) measures the relative reduction in infection/disease for the vaccinated arm versus the unvaccinated arm. A perfect vaccine would eliminate risk entirely, so VE = 1 or 100%. This can be calculated from the risk ratio, incidence rate ratio, or hazard ratio.
3) Vaccine efficacy of 50% roughly means you have a 50% reduced risk of becoming sick compared to an otherwise similar unvaccinated person. Or you have a 50% chance of becoming sick given that you were exposed to enough infectious virus to make an unvaccinated person sick.
What is the best endpoint for Phase 3 vaccine trials? Disease of any severity? Severe disease? Infection? Most trials have selected the first, but this @PostOpinions piece is critical of counting mild cases. A thread in defense of what is being done. 1/6 washingtonpost.com/opinions/2020/…
First, this has been the plan for a while now. The WHO target product profile for covid vaccines was published in April, and lists that efficacy can be evaluated with respect to disease, severe disease, shedding, or transmission. 2/6 who.int/blueprint/prio…
Similarly, the FDA's guidance to industry was released in June. It states that either disease or infection are acceptable endpoints. 3/6 fda.gov/media/139638/d…
Even though vaccine efficacy is estimated by the incidence rate ratio VE=1-IRR=1-(m1/T1)/(m0/T0), the underlying test is an exact test for a single binomial proportion. The proportion in question is what fraction of total endpoints are in the vaccine arm p=(m1/(m1+m0)). 2/6
Under 1:1 randomization, we have roughly equal follow-up time across both arms (T1=T0). If our null hypothesis was VE=0, we would expect p=50%, with the same number of events across arms. This null can be expressed as a function of the follow-up time p=T1/(T1+T0). 3/6
Informative op-ed on vaccine EUAs from @ScottGottliebMD and Mark McClellan, but I still have questions about the process. Namely, how would an EUA impact our ability to collect the additional data we need to support broader use of a vaccine? 1/4 wsj.com/articles/how-e…
“The EUA process gives the FDA the power to require additional data as a condition of authorization. This is a virtue of the pathway... As data supporting safety and effectiveness grow, the FDA will have a stronger basis to make the vaccine available more widely.” 2/4
I haven’t seen enough discussion about how to collect additional efficacy data after an EUA is issued. Will randomized trials continue in populations not covered by the EUA? Will people want to participate? And what is the impact of an EUA on other products in development? 3/4
The language around vaccine EUAs is vague. “Benefits outweigh the risks.” Risks vary across populations. The interview seems to hint at an EUA for a high-risk population, rather than the general population? Maybe healthcare workers? (I am speculating.) 2/7 fiercepharma.com/vaccines/even-…
But remember that the risk profile has not changed since the FDA issued their June guidance, and published many op-eds stating their plan. 3/7 washingtonpost.com/opinions/fda-c…
What I don’t want:
- A return to stay-at-home orders
What I do want:
- Major investment in proactive solutions that allow us to return as close to normal as possible
What do proactive solutions look like:
- Widespread testing
- Rapid diagnostic tests (imagine having a 20 pack!)
- Robust tracing to detect new infections
- Support for safe isolation
- Better public education and messaging
- Improved ventilation
- Increased mask-wearing
Proactive solutions can decrease the reproductive number without requiring the general public to stay at home. This also lowers the threshold for how many people we need to get vaccinated to achieve control.
From the UK, evidence of improved survival in the ICU. A reflection of better patient management (plus maybe also increased availability of ICU beds). But this is why I never understood arguments for "front-loading" infections. Surely patients are better off now than before.
This is a response to people who said that flattening the curve was just delaying inevitable deaths. But of course it has also bought us time to improve care in big and small ways. And we are learning every day, with potentially better treatments still to come.
Who said this? From this letter:
"I expect that when we count...deaths from COVID-19 in each country in 1 yr, the figures will be similar, regardless of measures taken.... A lockdown only pushes the severe cases into the future —it will not prevent them." thelancet.com/journals/lance…
Excerpts from this article on challenges with contact tracing efforts in California. "Some simply can’t be bothered, but more often people decline to participate because they are worried about wage loss, deportation or stigmatization." nytimes.com/2020/08/09/us/…
"Over the most recent seven-day period, less than 60 percent of people who tested positive for the virus agreed to an interview... Response rates vary, but in LA, they are so worrisomely low that the county is now offering $20 gift cards to people who complete an interview."
"Contact tracing is not just reading from a script, though every tracer is given one. It is not just calculating the number of days a person must isolate, or tricks of the trade, like that it's better to call older people earlier because the younger ones tend to sleep in."
This article touches upon many of the concepts we have been thinking about over the last few years. How do we evaluate vaccines/therapeutics during public health emergencies? RCTs are tricky but important! A few key points... 1/8 nytimes.com/2020/08/04/hea…
There is genuine uncertainty about the value of convalescent sera for treating severely ill patients. This is best addressed by randomized controlled trials. But patients are unwilling to enroll in trials because they can receive sera through expanded access programs. 2/8
"A spokeswoman for the FDA said that the expanded access program was meant to bridge the gap until trials could get underway and “was never intended to substitute for randomized clinical trials, which are critically important for the demonstration of efficacy.”" 3/8
Really appreciate this from @StevenSalzberg1. There are important debates about how we do (or do not do) things differently during emergencies, and I think it is good for the public to see the pros and cons laid out.
My colleagues and I have spent a lot of time thinking about these issues, and the rules are not set in stone. There are settings where there is more flexibility. who.int/docs/default-s…
But what features of Covid vaccines convince me that we need Phase 3 trials? (1) Insufficient safety data. We only have reports from relatively small trials, and so side effects are not well characterized. statnews.com/2020/07/27/cov…