Multiple issues with the widely quoted NEJM Israel study on boosters
Long thread👇
1. Authors report a lofty reduction in infections & severe cases by a factor of 11.3 & 19.5 in the primary analysis, where rates are compared between boosted & non boosted groups.
But ...
1/
Long thread👇
1. Authors report a lofty reduction in infections & severe cases by a factor of 11.3 & 19.5 in the primary analysis, where rates are compared between boosted & non boosted groups.
But ...
1/
https://twitter.com/walidgellad/status/1438262619882803200
2. In secondary analysis, this factor is down to 5.4. Secondary analysis compares rates within the SAME group, by timeframe. This is more believable not only because comparison is within the same group, but also because we know higher antibody levels reduce infection rates.
2/
2/
Note: secondary analysis is available ONLY for infections, not severe cases.
In other words, we do not yet know if this 5.4-fold reduction in ‘infection’ will translate to reduction in hospitalisation/death later.
3. No mention of number of people who were hospitalised.
3/
In other words, we do not yet know if this 5.4-fold reduction in ‘infection’ will translate to reduction in hospitalisation/death later.
3. No mention of number of people who were hospitalised.
3/
4. Ultimately, results depend on testing rate in the study population.
Not all severe cases are the same: a respiratory aate of >30 is also classified as severe. This can be missed by someone sitting at home, and can only be detected if the person seeks medical attention.
4/
Not all severe cases are the same: a respiratory aate of >30 is also classified as severe. This can be missed by someone sitting at home, and can only be detected if the person seeks medical attention.
4/
People who receive a booster are likely to:
1. Be excessively cautious about avoiding exposure and
2. Less likely to go for testing because they feel overconfident and might ignore mild symptoms.
5/
1. Be excessively cautious about avoiding exposure and
2. Less likely to go for testing because they feel overconfident and might ignore mild symptoms.
5/
5. The supplementary table (see pic) shows that testing rate immediately dropped by 30% from the very day after taking booster (see graph).
This is clear evidence of behaviour driving the results rather than disease.
6/
This is clear evidence of behaviour driving the results rather than disease.
6/
7. Confusingly, they don’t mention the number of people who got booster shots vs those who did not.
In other words, there is no denominator provided. Instead, they use person-days. I would want to know absolute no. of people, as well as frequency of tests done in each group.
7/
In other words, there is no denominator provided. Instead, they use person-days. I would want to know absolute no. of people, as well as frequency of tests done in each group.
7/
8. The comorbidity or behavioural pattern of people who came for boosters is unavailable.
Since it is a non randomised study, results can be affected by several such confounding factors.
e.g. we don’t know if the booster group had low baseline exposure risk (healthier)
8/
Since it is a non randomised study, results can be affected by several such confounding factors.
e.g. we don’t know if the booster group had low baseline exposure risk (healthier)
8/
Summary:
One must be careful while interpreting observational studies that are non-randomised, particularly when the effect shown is rather too large & too sudden to be true. Long term studies will tell us more, but will likewise be subject to multiple confounding factors.
9/9
One must be careful while interpreting observational studies that are non-randomised, particularly when the effect shown is rather too large & too sudden to be true. Long term studies will tell us more, but will likewise be subject to multiple confounding factors.
9/9
Dr @jeremyfaust has also analysed this paper. He has raised important and similar concerns as I have listed above.
Reassuring to know critical thinking still exists. @celinegounder
see thread above👆 for context
Reassuring to know critical thinking still exists. @celinegounder
see thread above👆 for context
https://twitter.com/jeremyfaust/status/1438248705124184064
@threadreaderapp please unroll
• • •
Missing some Tweet in this thread? You can try to
force a refresh
