Thread by @alexandrosM

Is Ivermectin enthusiasm research founded upon fraudulent research?🧵

This investigation from the BBC, citing research by @GidMK @sTeamTraen @JackMLawrence @K_Sheldrick and @jamesheathers, claims the answer is yes.

Let's dive in and see what we find. bbc.com/news/health-58…

I don't tend to list credentials, but in case this is read by academics, and this makes a difference, I'll mention that I have a PhD in Computing. Google says that the papers I wrote in my ~3 year publishing run have been cited 865 times and that my h-index is 11.

The BBC article is laced with the usual anti-ivm talking points I've addressed in other threads, and I will assume that they have been added by the journalist. In this thread, I'll focus on the fraud claims, which are sourced to the investigations of the group.

The researchers are champions of transparency in academic research, and advocate for a meta-analysis standard that mandates review of patient data from each study, a position which they stated in their recend correspondence in Nature. nature.com/articles/s4159…

Despite this advocacy for data transparency, they have not published the data that they have used to make their claims of fraud. As a result we'll have to use what fragments they have published on Twitter, blogs, and the press.

I had been told by one of the researchers over 3 weeks ago that the data would be public "soon", but it appears that they have gone to the press without publishing that data anywhere. It would be really good if they could correct this as soon as possible.

Given that this researcher appears to have blocked over 12% of my followers, I will be posting screenshots of his tweets whenever I reference them.

The BBC article says that "The group of independent scientists examined virtually every randomised controlled trial (RCT) on ivermectin and Covid". It mentions they examined 26 studies, of which 6 were observational, so I infer that they looked at 20 RCTs.

The ivmmeta website lists 31 RCTs. To make a good faith assumption, let's remove preprint RCTs and perhaps also Cluster RCTs that may be considered inferior. (Chahla, Together, Buonfrate, Bukhari, Biber, Gonzalez, Huvemek, Seet). This leaves us with 23 RCTs.

The researchers have raised significant issues with 3 RCTs: Elgazzar, Samaha, Niaee. I've also heard that there is a potential issue with another study. Elgazzar and Samaha have been removed from ivmmeta, so if we add them back, it looks like 4/25 studies must be thrown out.

Wow. That's 16% of all Ivermectin RCTs. Is that high or low though? One of the authors certainly seems to claim that it is extremely unexpected. gidmk.medium.com/is-ivermectin-…

When looking to see what level of bad research should be expected as a baseline, a BMJ editorial proposes a baseline of 20%. Indeed, all three RCTs publicly implicated are from the middle east, where there is a *very* high incidence of fraudulent research. blogs.bmj.com/bmj/2021/07/05…

Thus, both in geographic origin and overall prevalence, it appears that the compromised RCTs conform to the baseline pattern described in the BMJ editorial. It is unclear why the researchers did not establish a baseline of expected research to compare their findings to.

The authors however also claim to have selected 6 observational trials to investigate. Given that they have not pre-announced the studies they would investigate, it's hard to know what to make of the claim that they chose "particularly influential observational trials".

The one observational trial discussed in the article, by Samaha et al, is small (includes just 100 patients) and there is no indication of what would have made it "particularly influential", casting doubt on the claim as written.

There is, however, one other claim: that for 14 of the 26 studies investigated, the authors declined to share the patient data. I'm not sure what the norm is around such data-sharing. There is no information on what background communication was involved.

I would find it very surprising if authors of randomized controlled trials were forthcoming with data on their trials, given the strict recent privacy legislation, (HIPAA, GDPR, and global equivalents) but I admit I am not an expert here.

It is however somewhat naive to say that anonymization that leaves intact place & time of the study as well as age and gender of the patients as well as other health information such as co-morbidities can be done without great cost. For why, see linked 🧵

Setting that aside, and putting myself in the shoes of a researcher on the receiving end of such a request, I might try to understand who I'm interacting with. If, for instance, I were to look up the most senior of the group, I might find this:

If I were to look up the person with the highest public profile, I might find this:

As a busy medical researcher in the middle of a pandemic, I'm not sure it would be particularly surprising if there was no willingness to engage people who appear to have an agenda.

It should be said that the other 3 members of the group have far more reasonable public communication in the pre-Elgazzar retraction period, which is where I focused my search. It must be said that as far as I looked, @JackMLawrence maintained a high-integrity & honest voice.

At this point I must separate two concerns. Any work being done to show fraudulent research should be praised regardless its source. I am personally grateful for the work they put in, and consider it a service to science and humanity.

However, the claims being made on the basis of those findings, which for the moment appear to be under the expected average, are far less defensible, and honestly within the realm of the behavior they accuse others of. Let's walk through some examples.

For instance, Dr Sheldrick makes a sweeping claim, implicating the entirety of the research body:

This claim is particularly strange, in light of his praise of the Mahmud study, and the particularly severe words he had for anyone who might cast doubt on its quality:

Another of the researchers, interpreted the finding of the Samaha study as a blow against the usefulness of ivermectin, and not the first: gidmk.medium.com/is-ivermectin-…

It's hard to know what to make of these claims, and how they could possibly be justified. Science uses meta-analyses to draw conclusions over the entire body of research on a particular question, and the authors are on record requesting *more* strict criteria for them.

Why they believe that without maintaining any of the rules of a meta-analysis they had a footing to make claims about the underlying matter of ivermectin is a question that I believe they should examine. Their claims are extensively covered, and the implications are far-reaching.

I don't believe I will be able to further publicise this analysis, so if anyone wants to write it up in any other form, for propagation, feel free to do so. I would appreciate, but do not require, attribution to this thread.

A worthwhile question.

From the comments, another perspective I'm increasingly feeling compelled by.

At the risk of veering off topic (the main body of the thread was done a few tweets ago, feel free to look away), I've been wondering if entirely different approaches would be possible to follow for faster and more reliable answers to hard questions.

One of the errors being made repeatedly in public communication by members of the team of researchers is to endorse the "bright-line fallacy". This is critical in understanding their interpretation of the facts.

Over the weekend I wrote a second part to this thread, proposing an alternative explanation for the data pattern this team has been working with. (surprise! I got called "racist" for doing so)