Thread by @mahesh_shenai

1/ IMPORTANT THREAD: @ 5-11 VRBPAC Members saw “scenario analysis” of vax BENEFIT vs RISK, by an FDA epidemiologist. The math is simple, but the scenarios considered were FLAWED & NARROW. This 🧵recreates the model w/ new and more germane scenarios.

1b/ SPOILERS FIRST:
In AVERAGE INCIDENCE SCENARIOS , vax prevents CASES in COVID-(N)aive on par with vax “related AEs”, AND serious AEs/death generally on par w/ C19 hosps saved.

1c/ For COVID-(R)ecovered, all AEs gneerally exceed C19 cases/hosp/death prevented, in all but most extreme assumptions (occurring ALL AT THE SAME TIME, sustained throughout the entire period).

1d/ The original FDA presentation can be found here. fda.gov/media/153507/d… The analysis was authored by the FDA Office of Biostatistics and Epidemiology, not Pfizer.

2a/ First, lets understand the FDA model. In essence, they modeled incidence rates (cases, hospitalizations, ICU, death) against the same of myocarditis ONLY. They compared these outcomes in “base”, “peak”, and “low case” instances.

2b/ However, there were some flawed assumptions in the model. First, the “base” case selected around 9/11 actually represented near peak incidences (250/week, 2.5x average) rather than “average” incidence, overestimating cases prevented.

2c/ They also only considered only the adverse effect of myocarditis, and not the totality of other adverse effects that may occur.

2d/ They also did not consider the COVID-(R)ecovered which may account for up to 40% of children 5-11 yo. This group has an EVEN MORE reduced baseline incidence of reinfection.

2e/ Even with these flawed assumptions, they demonstrated Scenario 3 in which myocarditis (alone) hospitalizations exceeded C19 hospitalizations and ICU admissions, particularly in MALES.

2f/ This alone argues against any enforcement of a “mandate”, because it does not guarantee benefit > risk for ALL SUBGROUPS in ALL scenarios. But still, let us comprehensively consider more scenarios:

3a/ Here, we consider a similar mathematical model. But include COVID-R, ALL types of reported AEs (not just myocarditis), & a base case closer to the overall average across the pandemic (100/100k-week cases). All rates converted to /1M-6mos to compare to FDA scenarios.

3b/ Overall, the rate of cases/hosp/death saved is the (VE x Incidence).
For COVID-R incidences are additionally discounted by 1-%RR of natural immunity protection. VE for COVID-N is set at 90%, but for COVID-R, its varied.

3c/ Total AEs are difficult to quantify, because it varies and covers a large spectrum. For all related AEs (mostly mild/moderate), we can take from the Pfizer 12-15 yo trial (Frenck Jr., NEJM) = ~3% (pubmed.ncbi.nlm.nih.gov/34043894/ )

3d/ But Frenck Jr. et.al. studied only 1600 vaccinated kids, and was too small to detect rarer event signals. “Serious” AEs generally require hospitalization, “severe” AEs fall short of hospitalization.

3e/ cdc.gov/mmwr/volumes/7… Another CDC study of MMWR found a 1/1000 VAERS event rate, of which 10% were serious. So we assume a “serious adverse” event rate of 1/10,000 (0.01%), and a 1/1M mortality.

3f/ These are fairly conservative values, but have many caveats due to reporting methods and limitations. And, this is very much UNKNOWN in the 5-11 yo range. But use for now, and qualify later.

4a/ BASE CASE: We assume 1000/1M cases/week (this is slightly higher than the average from 3/20-10/21). Assume NI RR=75% and COVID-R VE=50%, but COVID-N VE=90%.

4b/ BASE CASE: For the COVID-N, C19 cases prevented on par with all related AES, & serious AEs/death similar for C19 hosp/death (101 vs. 72/1M-6m). For COVID-R, AEs > prevented events across all severity.

5a/PEAK CASE: This assumes 3500/1M-6m, all other variables the same. For COVID-N, prevented C19 cases, hosp, deaths >> all AEs. For COVID-R, Serious/Death AEs still > C19 hosp/death.

5b/ PEAK CASE: Of note, this scenario alone is unrealistic, because these incidence levels would have to be sustained across the ENTIRE 6m, to actually accrue these numbers!

6a/ Weak NI Protection: Literature says COVID-R receive 75-90% risk reduction for reinfection. For this scenario, let us assume only 25% risk reduction. All other assumptions same as “base case”.

6b/ Weak NI: All related AEs still > cases prevented for COVID-R. Serious/Death AEs still >> C19 Hospitalizations/death (100 vs. 29)

7a/ INCREASED COVID-R VE: Several studies have shown COVID-R vaccine efficacy ~50%, but as low as 18-33% after a few months. What if COVID-R VE is the same as COVID-N VE (90%)?

7b/ INCREASED COVID-R VE: All related AEs still > cases prevented for COVID-R. Serious/Death AEs still >> C19 Hospitalizations/death (~100 vs. 18).

8a/ WORST CASE SCENARIO: So, what if we put all unfavorable scenarios together? Peak incidence, weak NI and increased COVID-R VE, sustained across 6 months ?

8b/ WORST CASE SCENARIO: In this case, COVID-R will have more cases, hospitalizations and deaths prevented then serious vax AEs/deaths (184 vs. 100 hosp, 8 vs. 1 deaths). This is virtually an impossible scenario, however.

9a/ We assumed that COVID-R AEs are the same as COVID-Naïve AEs. There may be some evidence that COVID-R AEs are worse. If so, the presented analysis shifts further away from vaccination.

9b/ Similarly, if you feel that this analysis has “overestimated” AEs, discount accordingly. For COVID-N, it may shift the conclusions. For COVID-R, you would have to believe 1/10th of the assumed AE rates to break even!

10/ And, we have not considered “unknown” risks that may appear beyond the follow-up periods of studies currently available. This point was made well during the VRBPAC.

11a/ The point of a model is NOT TO PREDICT THE FUTURE. It is to understand how certain inputs change a result, and then assess the likelihood of certain inputs.

11b/ We have a good sense of the C19 morbidity that may be prevented with vax. NI and VE are known, but changing variables. AE estimates are more uncertain, but small changes in AE rates could significantly alter this analysis (both ways).

12a/ I agree with approval for 5-11yo as an OPTION only. There are a subset of kids that likely will benefit from the vaccine – COVID-naïve, with specific comorbidities. There are also subgroups that will NOT.

12b/ The POINT here is different SUBGROUPS and SCENARIOS can quickly tilt the risk-benefit balance in this age-group, precisely because the rates of severe morbidity is so low.

12c/ Any policy or guideline that can be interpreted as a mandate is the result of policymakers and leaders FAILING to do the hard work of understanding data and nuance. EMBRACE PRECISION.

13/ Thank you for reading. I respect any comments, corrections, critiques. Appreciate the RT to get the message out!