Thread by @alexandrosM

The Real Scandal About Ivermectin🧵

Or, "Rigor for thee but not for me"

Let's walk through @jamesheathers latest article in The Atlantic to see if he and his collaborators have been paying attention to the criticism of their work.

theatlantic.com/science/archiv…

The subtitle is already setting the stage: "Claims about the drug are based on shoddy science—but that science is entirely unremarkable in its shoddiness."

If I am reading correctly, this is saying that both "the science behind ivermectin is shoddy" and "that's pretty typical".

Does that mean that he'll come out and just say "don't trust most/any medical science"? I find that hard to believe, but let's see what we see.

"Ivermectin is an antiparasitic drug, and a very good one" DRINK

"This promotion has been broadly criticized as a fever dream conceived in the memetic bowels of the internet" DRINK

"and as a convenient buttress for bad arguments against vaccination" DRINK

"This is not entirely fair." HOLD THE DRINK, this guy may be reasonable?

"Perhaps 70 to 100 studies have been conducted on the use of ivermectin for treating or preventing COVID-19; several dozen of them support the hypothesis that the drug is a plague mitigant."

Several issues here: First, these error bars are off the charts.

Second, what is a "plague mitigant"? Can't you be more precise?

Third, by phrasing it like "several dozen" you give the impression that maybe half, maybe less are positive. That's a false impression. Nevermind the "reasonable".

Ah, we start getting dicey: "one physician and congressional witness did recently, that “people are dying because they don’t know about this medicine.”"

"Recently" does a lot of work here. When was this? On June 1. Before any of the ivermectin "fraud" claims were even public.

How do I know the exact date? Well, I may or may not have paid for a transcript to be written of the exact podcast that quote is from. And of course much of the nuance was ignored in favor of the zinger. betterskeptics.com/transcript-cov…

"The problem is, not all science is worth following."

Well, that's something we agree on. Especially if we're talking about The Science (tm).

Next up, we're introduced to the team of "forensic peer reviewers". I'll leave aside for the moment whether this team actually counts as "peers" of the people they're reviewing, and focus on the kinds of "fatal flaws" they claim to find.

So, let's see what their selection criteria is. Apparently they've examined "about" 30 studies. Is it unclear how many studies they've examined? Is this one of those "two dollar" words?

They're apparently focusing on RCTs - OR - "non-randomized ones that have been influential".

Something tells me that "influential" is another one of those "two-dollar" words.

Let's go back to basics: Scientists do meta-analyses to make broad statements about research literature on a particular topic. These have inclusion/exclusion criteria, ideally pre-registered.

These criteria can then be followed by third parties so that one can come to the same set of studies. Now you tell me, can you follow that text and come to the set of "about" 30 studies this team has examined? And in any case, why is this list not shared with us after the fact?

Next up, we get a sense of the method that is being followed. They reach out, making unspecified requests. In "their opinion" 5 papers have big issues. Also, they claim one study has been withdrawn on the basis of "their" work.

Here's the thing: The study was withdrawn as a result of the work of 1 of these researchers, *before* the group was put together. The other 4 don't seem to have been withdrawn, and given that they tell us nothing about their methodology or findings, we just have "their opinion".

Guess who else has been reaching out to researchers, asking for their dataset: Me.

Which ones? The ones writing the article.
What have I been asking for? Their data.
When? 6 weeks ago.
Do I have it? No, but the person I asked (@gidmk) blocked me.

https://twitter.com/alexandrosM/status/1445184227020996610?s=20

But I asked again.
Who? The author of the Atlantic piece, almost 3 weeks ago.

Did I get anything? Nope. His answer was "I run a startup, we'll do what we can".

https://twitter.com/alexandrosM/status/1446280739238993925?s=20

I also asked a third member of the team, @JackMLawrence, a few days later. No response.

https://twitter.com/alexandrosM/status/1446789736933122048?s=20

I've also asked a fourth member of the team. @K_Sheldrick. Do I have any data? No. At this point even the most well intentioned has to wonder why this team is not publishing its dataset of "about" 30 papers.

https://twitter.com/alexandrosM/status/1446832811852242955?s=20

Here's where things start to get really strange. In a recent BBC article describing this team's findings, they had apparently looked at 26 papers, and found major issues with 10. So now, with "about" 4 more investigated, they've got 5 papers that they're not willing to accuse?

For more details on that BBC article and my analysis of it, see this thread, though I'm getting a feeling we'll be revisiting it later on in this thread:

https://twitter.com/alexandrosM/status/1446823118241107968?s=20

Next paragraph, the author goes over the Elgazzar and Carvallo studies, which most everyone agrees have big issues. Then we get to the Samaha trial, with 100 participants. Interesting number, precisely at the lowest bound. Just 10 citations, none a meta-analysis.

Next is their particular analysis of the body of research. They start by assuming all their work is valid, and attempting to steelman the opposition. Let's go with it and see what's coming next.

Ah, we reach the crux of the argument. Read this clipping because we'll need to go through it slowly.

First, he claims "it's likely we missed other problems". Just imagine if anyone makes this kind of claim for, e.g. the vaccine trials. This isn't how any of this works indeed.

Next, he claims that they don't have enough time to do the work in depth. Unclear how this should factor into our thought process.

Then comes the real zinger: "We don’t pick papers to examine at random, so it’s possible that the data from the 30 papers we chose are somewhat more reliable, on average, than the rest". Yup. This is what we're getting for why they're smearing hundreds of working scientists.

"It's possible the papers we chose are more reliable".

Or, hear me out, given that "influential" might be a proxy for "outlier result", you may have picked the papers with the *worst* data. At least I'm providing a mechanism. What's yours?

Wait. He admits that the most extreme results are the ones they're most sure are bad. This is congruent with a hypothesis that the rest of the papers have *better* data (and positive results). Why he claims that the data they looked at is *better* than the rest is beyond me.

BREAKING: The author saw this thread. His response? To "accuse" me of running ivmmeta.com. If I did, I would be proud of it. But I don't. However, we get a fantastic clue of the kind of detective skills this crew is working with.

Moving on, the author launches into a just-so story about how the pandemic gave rise to shoddy science. Do we get to apply this story to any other interventions during the pandemic? Naaah. Let's stick with ivermectin for no apparent reason.

But wait - didn't the author say that this level of quality isn't uncommon? Is he saying now that it *is* uncommon? Has he ever investigated any other medication with this level of attention to detail? Ah, someone asked. Answer? Nope. So whence the claims?

https://twitter.com/mmpaquette/status/1446291627207798796?s=20

Well, we come to the same page again. I also do my forensic work on the work of this group without getting paid. I do it because I feel it needs to be done. What is their response? Sneering. Rigor for thee but not for me.

They then say various things about their methodology until we get to this gem again. This group just can't seem to understand that they can't just smear hundreds of scientists, based on *unreviewed*, *unpublished* meta-analysis, that doesn't meet even the most basic standard.

They do seem to have taken on board my persistent comments that they should talk about the baseline of fraudulent research (after mocking me for it for weeks) though. And their conclusion is that their findings are unremarkable. Assuming they're true. And yet, claims they make.

Good quote (taken from a reference I found for them. I'll accept co-author credit, James). Author says we should start from a baseline of "I don't believe you". Very well then, @jamesheathers. I don't believe you until you definitively show me otherwise.

This quote, however is gold. I'm having a hard time believing the author isn't getting a heavy sense of self-consciousness. What he writes here, without substantiation, applies to himself and his teammates, with substance. Word by painful word. I can't believe what I'm reading.

This is his final paragraph, and I'll just leave it here without comment.

I asked the author to discuss live. His response? Irony.

In case you've just landed from outer space, you can rewind to my analysis of the first article from this group in the BBC here:

https://twitter.com/alexandrosM/status/1446271495454429215?t=58GgyEr0yLVYcKBri9U9Lw&s=19

And the followup, where @k_sheldrick called me racist for proposing a better fitting hypothesis for their data that, despite their not publishing them, I reverse-engineered from their well-sculpted statements.

https://twitter.com/alexandrosM/status/1446823118241107968?t=58GgyEr0yLVYcKBri9U9Lw&s=19

This, sadly, is where we are in this pandemic. Rigor demanded from those we don't like, and not even a preprint offered from the defenders of the establishment. The irony is so thick it doesn't even blend.

Perhaps the best way to close this thread is to remind you all of the #1 rule of this pandemic. Good night and good luck.

https://twitter.com/alexandrosM/status/1449764288571207696?t=58GgyEr0yLVYcKBri9U9Lw&s=19

Fast-follow post-script. I've offered the author to discuss this article with him on a live stream, with a format of his choice. We'll see what he says.

https://twitter.com/alexandrosM/status/1453963920360701957?t=58GgyEr0yLVYcKBri9U9Lw&s=19

Morning update: a new hope.

https://twitter.com/alexandrosM/status/1454100262776233989?t=GB7b4X9QFUxItA93OCe0ng&s=19