Thread by @ewanbirney

There are some good and bad hot takes about the Oxford/AZ vaccine going around - I'd like to add my view which I hope is more light than heat.

Context: I am not a clinical trials expert, (rather a computational biologist / genomics expert) and I have an interest because I am trials participant in the Oxford/AZ trial.

The headache in this trial reporting is that a dosing error/operational change made a different dosing regieme which looks more effective. This has complicated analysis, reporting and communication.

(One aspect here - the way the data in particular on randomisation is generated and stored there is basically no way to "cook the books" seriously - the one way is repeated sub-group analysis which I will come onto)

We only have the press release (because the company has to release as soon as the triggers are met) but we will I am sure get a preprint quickly. This is frustrating but it is standard practice. The alternative is a large set of people who know the result before others

This is not good for transparency and it is not good stock market stuff.

Stepping back, there are three things you want to know in a trial. #1 is it safe. #2 - does the intervention work *at all*. Then #3, if it works and is it safe, how well does it work?

Going through this; #1 is definitely true (to the limit the trial can test, but in this case we don't need to worry about infection rate; everyone in the trial is part of safety). The first - and perhaps most important - tick.

The next, #2, it also definitely true. If one discarded the half-dose, full-dose "error" it would be true (using @wolfgangkhuber's back tracing of likely counts). The half-dose, full-dose by itself would be true. Them jointly is even more true.

Then, finally we get to efficacy - how well does it work. Here it gets murkier. But before we get into this remember that the endpoint here is symptomatic infection reported after weekly testing for infection.

(the Oxford/AZ team will presumably have data also on fully asymptomatic infection).

This endpoint is the "deepest" into the biology of the virus that I know of the 3 trials reported yet; the other trials went for symptomatic infection as the trigger, then confirmation by RT-PCR (other way around) is my understanding.

In the press release the Oxford/AZ group said that none of the vaccine treated individuals were hospitalised or died; we don't know the sham/placebo arm numbers (I presume it is low) but 0 on the treated arms is the best result one could get at this point in the trial

Back to the efficacy numbers; the half-dose full-dose trial has seemingly higher efficacy than full-full. Now there is a shift around of subgroups / countries in the full-full but the signal is all coming from the half-dose full-dose improve (4 vaccinated plays 34 sham)

Now this is a sub-group analysis (which immediately triggers concerns) but it is not a post-hoc sub-group creation (which is the worst thing to do). It is a "natural experiment" addition to the trial.

This randomisation is within the age group (18-55) and within one country (UK) and it is likely that the mirror full-full dose of age group (18-55) and country (UK) will be fine to use the dosing regieme shift as a natural experiment.

(One might get into "near Oxford" vs "far from Oxford" confounders but remember this is about a shift in efficacy, not a "does it work" and randomisation happened *throughout* - the HF has a matched sham arm).

Clearly we need the papers (all 3 trials in fact) - and clearly it would be cleaner if the variable dosing arm was randomised explicitly but it reminds you that the immune system is ... not something we understand. Welcome to science!

As we will likely give the vaccine in a variable way in some roll out (in particular with HCWs getting it early, there will be another broad early group). It is worth noting all the trials are likely to continue for a fair while longer (so each of these estimates will improve)

In summary: we now have 3 SARS_CoV_2 vaccines which, on summary reports of early data, work. We should be overjoyed. Understanding efficacy and details is important, but will come. Scale up, logistics and roll out is also important; let's get to a better world as soon as possible