Thread by @GidMK

Lots of people have been asking me to take a look at this observational ivermectin trial from Argentina, and I thought it'd be a nice change from all the fraud so here's a bit of a thread 1/n

2/n The trial is here, and it is a fairly simple epidemiological trial comparing people who were given ivermectin with those who weren't on ICU admission/death using large-scale registries in Argentina
zenodo.org/record/5525362…

3/n I've done a brief check for fraud, and the study looks fine. Honestly, I have no real issues with this paper as is, it's just not very useful as evidence for ivermectin

Let's think about why that is

4/n In observational trials, we have groups of people who have, for whatever reason, been grouped into categories. In this case, doctors and patients decided whether they should use ivermectin or not

5/n This creates an inherent bias in the dataset - there are obviously going to be differences between people who are offered/decide to take ivermectin and those who don't

6/n And in this case, we KNOW the two groups were very different because the authors demonstrated that they were

This is what's commonly called confounding, and you can adjust (control) for it

7/n The problem here is that we can control for variables that we collect - the authors have a model with age, sex, and a couple of comorbidities - but we can't control for stuff that we don't measure

This is called RESIDUAL confounding

8/n One very common issue with studies looking at medicine prescriptions is that people who get more meds are often richer, drink less, and eat better than people who don't get the meds

9/n Unfortunately, due to limitations in the data collection, the authors did not have enough information to control for these factors, or any of dozens of other issues that might cause people to be less likely to die

10/n What this means is that it's hard to interpret the results. People who got ivermectin died less, but they might have been richer, or had less severe COVID-19, than people who didn't get ivermectin

We just don't know!

11/n This is why it's important to run randomized clinical trials - observational research is amazing for lots of things, but when it comes to measuring the impact of a drug it has some very significant limitations

12/n Unfortunately, those limitations have nothing to do with sample size. It makes no difference if you've got 10 or 10,000 people, if you can't control for biases your estimates will be just as problematic

13/n (And I say that as someone with datasets that number in the millions with exactly the same issues that this research has. It's not an easy problem to solve!)

14/n Anyway, none of this is the fault of the authors, who ran a simple epidemiological study. It's fine as far as observational research goes, it just doesn't change the evidence-base much if at all