Thread by @ewanbirney

A brief explainer on endpoints and efficacy on Vaccines as it's clear we're going to have a lot of chit chat on vaccines as a topic to discuss.

First off - let's step back and recognise that it is *awesome* that we have 3 COVID vaccines that are safe + work , and most likely more in 2021 (I suspect the Chinese vaccines will get regulated in more and more places; for a variety of reasons the Russian one will be complex)

This should frankly be enough. End of discussion - over to the eye watering logistics about vaccinating as many people across the globe as fast as possible. We're 0.06% into 7 Billion people as of today.

...but... people are going to compare.

(Context for the below discussion: I am *not* a clinical trials expert but I do know a number of experts; I am genetics and computational biology expert, and computational biology is a mix of data science and statistics, so I understand the shape of this discussion).

(Other context - I am a trial participant on the Oxford/AZ trial).

In Phase III clinical trials you aiming to understand two things (a) safety at a far larger number of people, so can get into rare(r) events and (b) efficacy, how well it works, traditionally in Phase III trials vs "current best practice".

On (b) you need to define what "works" means for your trial. This is described as the "endpoint"; you must declare your endpoints ahead of time and must do the analysis you say you will do on the endpoints.

Clinical trials nearly always have one primary endpoint (the main thing) and a series of secondary endpoints. Every endpoint you write down you need to show your working on.

Endpoints can be obvious ones ("did the person die from a heart attack") to pretty obviously ("did the person come into hospital with a suspected heart attack") to quite "upstream" in the biology ("Is the persons circulating LDL levels lower over 3 weeks at the end of the trial")

In this toy example, all these endpoints are about cardiac disease, and you can see that the last one probably has most "power" to see effects (because everyone is valid to test) but here you need to be confident that this endpoint (lower LDL) is really something of interest

(Side note; for the example chosen here, researchers are confident that lower LDL systematically tracks to less heart attacks but you can imagine this not so obvious)

As well as defining endpoints, you need to nominate one of them as "primary" which you will report against.

Armed with this language, let's look at the 3 trials; BioNTech/Pfzier, Moderna and Oxford/AZ. All had similar primary endpoints - numbers of people tested positive via swabs. For BioNTech/Pfzier + Moderna, my understanding is this was symptomatic individuals who then were tested

For the Oxford/AZ trial, everyone got tested every week (or at least; we were all sent home with test kits for every week) and sent the test in; if positive, then brought in for more study, including whether we had symptoms.

This makes sense as a primary endpoint - the headache in an infection trial that is trying to be suppressed in your countries is that not so many people will get infected, so you want to have something as close as possible to infection.

Unsurprisingly all the trials also listed hospitalisation with COVID and then death by COVID as secondary endpoints; in these cases they didn't know whether one would get good numbers.

The 90-95% headline number for Pfzier and Moderna, this messy 62%-90% depending on dosing for AZ is on the primary endpoint - swabs (and notice the subtle difference). Before we unpack this headline number...

.... note that for the hospitalisation numbers they all had very good efficacy - so good it can't really be quantified (confidence intervals go up to 99%) but small numbers.

For the everyday person the "does the vaccine work for me" probably means this hospitalisation number. Here - all 3 vaccines work indistinguishably well in my amateur reading, and I can sense clinical trial experts saying "you can't do comparisons unless in the same trial!".

(quite right - so many little and big things can be different, from access to healthcare to distance to testing centres that between trial comparisons is more like a narrative than an analysis).

To the primary - are you infected - endpoint. Here clearly Pfzier and Moderna work well. Oxford/AZ's full/full dose - it's main trial worked less well but note its endpoint is test first, ask if you have symptoms second.

The dosing error which lead to half/full dose was not well controlled, but very suggestive it does work in the 90% level (the human immune system is weird) and I hope they do more studies (Phase IV trials?) on this to nail it down.

And there are real differences in logistics. The BioNTech/Pfzier on has to be kept at -70C and its lipid bilayer is fragile (no shaking when thawed). Moderna's lipid bilayer is more robusts (-20C). Oxford/AZ is a full blown robust but harmless virus (Fridge 4C fine).

I would be happy for myself - or any of my family - to be vaccinated by any of these three vaccines. I don't think the comparison on efficacy should worry people (though nailing down the details will be good) and its mainly about logistics in 2021. Roll on the vaccines!