Thread by @GidMK

Another day, another systematic review and meta-analysis of the same ivermectin research published

This one is positive. I don't think it should be 1/n

2/n Study is here, and generally it looks fine - search strategy was decent, they followed most guidelines (I.e. PRISMA), and overall the methodology was pretty reasonable for the stated purpose
academic.oup.com/ofid/advance-a…

3/n In fact, it is amazingly similar to the other systematic review that I looked at recently, down to THE SAME DETAILS THAT ARE WEIRD

This whole thing feels like some bizarre deja-vu

https://twitter.com/GidMK/status/1407140602009985025?s=20

4/n Skipping over the other methodological stuff (which was VERY similar), there is still a worry about publication bias in this newer review. Potentially an issue, hard to exclude as a problem

5/n The authors did attempt to test for this (good!) and found no strong evidence for publication bias

So some improvement there

6/n But, as before, the devil really is in the details

This review included those same two studies that I mentioned before - Elgazzar and Niaee

This time, however, they were rated as at LOW risk of bias (i.e. high quality)

7/n Now, it's perhaps debatable whether these have some concerns or are at very high risk of bias. I think the latter. But I genuinely cannot see how anyone who read the studies could think that they were at low risk of bias

8/n Let's compare to another paper - Mahmud (2020) is a study that everyone would agree is at low risk of bias. It is just incredibly well done

Also, it found a benefit for ivermectin journals.sagepub.com/doi/10.1177/03…

9/n Here's how Mahmud describes randomization. It is incredibly detailed, including descriptions of exactly how allocation was concealed, and even descriptions of the placebo meds to ensure that blinding was maintained

10/n In contrast, here is ALL OF THE INFORMATION from Elgazzar and Niaee on randomization, allocation concealment, etc

11/n Elgazzar simply does not have enough information to make a reasonable assessment (this would usually be high risk of bias for me). Niaee is clearly better, but the information is still not nearly as detailed as the Mahmud study

12/n And as I pointed out in my other thread, Niaee had ~enormous~ differences between groups at baseline, which should automatically put it at high risk of bias in the field of randomization

13/n I could go on, but suffice to say that I genuinely do not understand how anyone could read these studies and consider them at a low risk of bias. Have a look for yourself, there are innumerable issues:
researchsquare.com/article/rs-109…
researchsquare.com/article/rs-100…

14/n And guess what happens if you exclude these two studies from the mortality meta-analysis (RE/IV model in Stata) in this particular study?

Suddenly, ivermectin has NO BENEFIT

Again

15/n Indeed, the entirety of the ivermectin benefit across all of the literature appears to come down entirely to these two pretty low-quality pieces of research

16/n Now look, take this all with a pinch of salt, rating of bias is an inherently subjective thing and I might not be right

That being said, it is extremely concerning that all of the benefit seen for ivermectin seems to come from just 2 studies

17/n It is not a coincidence that out of the three (!) ivermectin meta-analyses to be published in the last 14 days, the two that included these studies found a benefit and the one that excluded Elgazzar did not

18/n In the absence of new evidence (this meta-analysis doesn't really count), I reckon that the only reasonable stance is that we don't really know if ivermectin works, and probably should not be using it outside of clinical trials 🤷‍♂️