Thread by @RajeevJayadevan

Apples 🍎 to oranges 🍊 comparison alert

Ref. A recent small retrospective study on Covaxin showed “50% effectiveness”. See link for paper & comment

I saw @GargiRawat was unfairly criticised for reporting

See thread; it takes time to understand👇

1/16

thelancet.com/journals/lanin…

1. The original efficacy study on Covaxin was a randomised controlled trial involving over 25,798 people.

This showed 77.8% efficacy against symptomatic disease, 93.4% against severe disease, 63.6% for asymptomatic and 65.2% at delta variant. Had tweeted in detail earlier.

2/

A randomised study starts with 2 groups of people. One gets vaccine, the other gets placebo. They are observed “prospectively” that is looking forward-during a study period. Disease outcomes are measured & compared between the groups. The % difference is reported as efficacy.

3/

The first study (RCT) was done at a time when the Delta variant had not become dominant. RCT is the gold standard for measuring vaccine efficacy. It is also easier to do early during a pandemic.

(It was not based in a hospital where excessive viral load is the norm)

4/

2. The recent study was done on only 2714 healthcare workers who underwent covid testing during a severe delta surge. It was a community study.

Think of it like holding an umbrella ☔️(vaccine) in a torrential rain, the earlier study was the same umbrella in a light drizzle.

5/

3. The recent study had a different design (hence “apple vs orange”) which is called a “test negative case control design”.

This is not the same as a randomised controlled trial. The results cannot be directly compared to start with. It was a retrospective “field” study.

6/

4. People often compare the marks of two students who took an exam from the same school, & conclude that one student “did better than the other”.

These two covaxin studies do not work like that. We need to understand that first. (That’s a lot of biostatistics & epidemiology)

7/

5. The first study (RCT on covaxin) was done all over India in November and January. Delta had just arrived. Among 50 cases of delta found, 13 were in the vaccine group and 37 were in the placebo group, resulting in a vaccine efficacy against delta of 65·2% (95% CI 33·1–83·0)

8/

6. The second study was done from April to May - exclusively on healthcare workers in AIIMS Delhi.

2714 people who underwent testing were divided into two groups, test positive and test negative. The authors checked how many in each group were vaccinated and unvaccinated.

9/

They compared 1068 case-control pairs and calculated an effectiveness of 50%.

7. However, remember that these are healthcare workers who could easily have had natural immunity from past infections, many of whom were asymptomatic. Serology was not done to identify these.

10/

That being said, even if serology was done, it would not pick up people who were infected more than six months ago, because antibodies would decline below detection limit.

This means that both cases and control groups will be “confounded by” presence of natural immunity.

11/

🔺This confounding will invariably reduce the difference between the two groups.

In other words, if the control group (negative) had NEVER had past natural infection, then the vaccination group would have shown a GREATER protection by vaccine, higher % “effectiveness”.

12/

As I wrote earlier, this confounding factor is distorting the vaccine narrative, artificially exaggerating the decline of vaccine effectiveness, which in turn fuels the push for boosters.

(There is no real evidence of decline in effectiveness @ severe disease after 2 doses)

13/

In fact, confirming this further, the authors have stated that in the control group, 11% had reported prior infection; while in the test positive group it was only 3.6%.

This means that the control group had enjoyed “unseen natural protection” even without vaccination.

14/

Thus, the 50% result must be viewed after considering the baseline difference between the two comparison groups.

E.g. while measuring the height of two individuals, if the shorter person is wearing high heeled 👠 shoes, the height DIFFERENCE between the two will be SMALLER.

15/

Thus, unlike a randomised trial done in 2020 where the placebo group is more likely to be previously UNINFECTED, (and therefore immunologically UNPROTECTED) the control group in the new study is not so.

In other words, the control group “was wearing high heeled shoes”.

16/

Thus, the measured DIFFERENCE in infection rates (which is how vaccine effectiveness is still calculated) between vaccinated & vaccinated populations is getting SMALLER as the pandemic is almost 2 years old now. This factor must be considered while interpreting such studies.

17/

In fact, what matters in the months ahead will be to measure the severity of the disease among these groups, while acknowledging the narrowing immunity gap between them.

This was not possible because severe disease was not reported in this study. It requires larger numbers.

18/

In summary, reading “just the conclusion” of any study without understanding basic biostatistics and epidemiology will lead to erroneous and wrong conclusions.

It is unfair to criticise media who attempt to report scientific studies.

19/19

@GargiRawat @snehamordani @BDUTT @ridhimb @AfridaRahmanAli @NairPreetuTOI1 @kitaab31 @sciejourno @AshishOnGround @PIB_India @Anandans76 @Milan_reports @IMAIndiaOrg @jayalal10 @BShajan @MalathyIyerTOI @Nainamishr94 @RPrasad12 @drbabukv