Thread by @michaelzlin

1) The news has been reporting on this "expert" editorial opposing boosters.

Just read it. Shocked that anyone would use pre-Delta studies to argue against boosters now.

Plus blatant data destruction that I'll try to explain.

A complete Lancet fail.

thelancet.com/journals/lance…

2) The editorial takes the form of a metaanalysis of vaccine efficacy, but it's the worst statistical malpractice I've ever seen. Fortunately it's short (one figure of 4 panels), so hopefully we can get through it quickly and point out the major issues.

Panel A: What is this??? Studies are grouped by efficacy ranges of 50-80, 80-90, and 90-100 for disease, then their average efficacy for all disease and severe disease are plotted. Why??? I'll try to be polite but I'm upset by this. It's an insult to human intelligence.

Problems:
1. No need to bin, just plot severe disease protection vs all disease for each study.
2. Binning destroys useful things: ability to discern outliers, ability to see any relationship curve, ability to discern variability from study to study
3. Bin boundaries meaningless

Panel B. It makes some sense to segregate studies by primary variant to try to discern differences between variants. But the big problems are:
1. Most studies provide data on one variant only. Because studies differ in how they define cases and severe cases and in follow-up time,

that means comparisons of efficacy of vaccines variants are hopelessly confounded. Remember how PH officials said don't compare RNA vax to JNJ vax because the studies differed slightly in when and how they're done? They just threw that out the window to lump all studies together.

2. Most importantly, efficacy of all vaccines vs Delta done at different times after vaccination, averaged together, means nothing for efficacy of a particular vax vs Delta 9 mo later. And yet the authors use this data destruction to argue we don't have data of waning immunity!

Panel C. The data destruction gets worse and more egregious. Vax efficacies are grouped by vax type. But again: different study times (but almost all within 6 mo of vax), different study criteria, different variants, different study populations.

Differences are not mixed in the same way between studies, i.e. each vax type's efficacy will be influenced by geography, case criteria, ages, etc in a different way than the other vax types. It's data destruction derby again.

And, 1-dose J&J grouped with 2-dose AZ. Meaningless.

D. Finally addresses the question of whether immunity wanes over time. The claim is "late" is nearly as good as "early". What is "late" though? And is it only Delta?

We see they arbitrarily set early and late to be <3 and >3 mo after vax. The early and late bins are adjacent, so it will be hard to see diffs! Also the question is if people 9mo out need boosters, not 3mo out, so the late bin isn't late enough.
Also there's one non-Delta study!

Also we see some studies had measured efficacies at different times after vax, so these studies had used the same measures in one popn to discern decay in VE over time. But by dicing the studies apart and then mixing parts from studies together, the info is lost. It's data murder

Above I marked with red arrows some examples of VE decay within a study. Some of this is antibody decay, some is Delta, but it doesn't matter why for the case for boosters. All that matters is if current efficacy is lower than a desired level for either all cases or severe ones.

If someone did a study to look at trends over time, there's zero reason to dice it into parts then mix with parts from very different studies, if your goal is to ask if there is a trend over time. That introduces variability that the original study design had eliminated!

Now if someone wanted to ask you, what is the average efficacy of vaccines across different strains at any random point in time in a random population across a vague definition of cases, then you could do this data mashup. But that's not the question!

The question being asked now is if Delta right now poses an unacceptable risk of breakthrough to people vaccinated 6-9 months ago. By trying to obscure the evidence, the editorial does a grave disservice to public health. The writers and Lancet should be ashamed.

You'll see news stories on this editorial say things like "experts say boosters not needed" or "scientists state opposition to boosters". That's unfortunate, because a reading of this editorial shows the expert label is not deserved. And there's a dire lack of science here.

I really don't enjoy trashing bad papers, but multiple people are now throwing this paper around as authoritative, and it shouldn't be. It's surprisingly amateur. It makes me sad to realize that people who formerly had high positions in the FDA would be willing to back this.

The only comfort we can take is that two of the authors no longer work at the FDA. But the presence of those people at high positions appears now as further evidence of the sad state of talent in those institutions that were supposed to apply knowledge to protect our health.