I'm realizing that the insitence on Randomized Controlled Trials (RCTs) as the only evidence that matters when deciding if a medicine/supplement should be used, structurally biases against generics, over-the-counter meds/supplements, and those with few side-effects. Here's why:🧵
The first class of problems has to do with wide availability when the subject of effectiveness on a new disease is raised.

1. Cheap OTC generics with few side-effects get used a lot in an emergency, where word of mouth spreads, making it much harder to form a control group.
2. These substances, when there's a suspicion they can be effective in an important disease, will spark many studies all over the world. This means there will be many small trials, of varying protocol/dosage and study quality. This is a big problem for two reasons:
2a. Varying dosage/protocol means that the results will necessarily be suboptimal. Even if some studies use the perfect protocol, others won't

2b. Some studies will be poor quality and/or fake. Critics can focus on these to invalidate every other study by diverting attention.
3. However, that's why we have meta-analyses right? Well, for something this controversial, there will likely be many meta-analyses, allowing motivated actors to pick and choose which one they prefer.
3a. Meta-analyses can focus on only "statistically significant" trials, or only on RCTs, and in general play with inclusion-exclusion criteria. Setting the criteria too tight, allows some analyses to remove most of the data, concluding "no evidence exists to suggest...".
3b. As some trials get shown to be of bad quality, calls for retraction of other meta-analyses get made, delaying conclusions. For a meta-analysis to be updated, it would need resubmission that can take months, even though the update takes a few minutes.
3c. Meta-analyses can water down the results of the strongest trials by averaging them out with the weakest trials. When the appropriate dosing is not obvious, this can significantly dull the effects seen.
4. Of course we shouldn't forget the obvious: RCTs are arduous, expensive, and doing one that's big enough to be "convincing" is prohibitively expensive.
A napkin calculation of what it would take to do a study for a 70% effective early treatment for COVID turned up a $10m cost.
Naturally, a new medication that is not available to anyone before the trial has none of these problems, and since it's going to be patented, budget is not a problem. The sponsor can do small unpublished studies to optimize the protocol, and then do a big one to showcase effect.
And if there's only one study to speak of, none of the meta-analysis problems apply.

This is why Obama's head of the CDC, Thomas Frieden, writes about moving beyond the RCT: nejm.org/doi/full/10.10…
Simon's perspective is always on point. Sadly he's being hidden under a filter, so here he is for your enjoyment.
Fascinating point about the steep, stepwise shift in cost and complexity. For most things they will either be too weak, or overkill.
Meditating on this, I suspect what we need is a meta-analytical framework that doesn't penalize a hypothesis for having been studied *too much*, that takes dose dependence into account. I don't know if it exists, but if it does, I bet the word "bayesian" is somewhere in the name.
I've put together a proposal for how to approach this for exactly that class of medicines and supplements, by using the same technology we use to optimize ad delivery and self-driving cars, but I'm not a medical researcher, so take it with a grain of salt.

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Alexandros Marinos

Alexandros Marinos Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @alexandrosM

11 Oct
I want to investigate the Hector Carvallo situation. This 🧵is likely to become overlong and meandering as I'll try to figure things out in real-time, so if you want to help please tag along, and if you want "just the facts" best to just mute this one and wait for the summary.
I'm aware of 3 main articles I'll try to comb through, and I've read none of them closely. Please comment with other resources.

[1]: buzzfeednews.com/article/stepha…
[2]: rescue.substack.com/p/a-lifeline-f…
[3]: trialsitenews.com/ivermectin-war…
The buzzfeed article is actually quite confusingly written. It seems the summary of the issues Buzzfeed found with Carvallo's work is:

- Numerical issues
- Hospital record
- Provincial record
- Declining to share data (incl. with collaborator)
Read 16 tweets
10 Oct
This is strange...
September 14: "Journalist and disinformation researcher"
Today: "pursuing a master's degree in biomedical science"

Why edit to obscure the journalism background?
I very well may be losing my mind, but this @ggreenwald tweet comes to mind..
@ggreenwald Jack seems to be the kind of person that classifies Jordan Peterson as a "rightwing YouTuber"...
Read 4 tweets
9 Oct
Apparently Twitter has unfollowed me from @BretWeinstein.

Hey @jack. When I say I want to follow @BretWeinstein, I mean I want to follow @BretWeinstein, and it's not a matter up to Twitter's whims to decide that I don't. Capish?
Read 13 tweets
9 Oct
I've criticized Kyle's group for not sharing the data they use to make sweeping statements. I'll demonstrate why this matters:🧵

Take this quote:"Every randomised trial that found a statistically significant benefit for survival was fake or did not actually happen as described."
First of all, it's important to say that while the tweet is far more carefully worded, the quote offered to the BBC, and not corrected when the article is shared, is far less careful, and basically false:
That aside, what's actually being said, is that if the bar is set to:

- Randomized Clinical Trial
- Endpoint is survival/death
- Result reaches p<0.05

All the trials with the above characteristics are "fake or did not actually happen as described".
Read 31 tweets
8 Oct
Is Ivermectin enthusiasm research founded upon fraudulent research?🧵

This investigation from the BBC, citing research by @GidMK @sTeamTraen @JackMLawrence @K_Sheldrick and @jamesheathers, claims the answer is yes.

Let's dive in and see what we find. bbc.com/news/health-58…
I don't tend to list credentials, but in case this is read by academics, and this makes a difference, I'll mention that I have a PhD in Computing. Google says that the papers I wrote in my ~3 year publishing run have been cited 865 times and that my h-index is 11.
The BBC article is laced with the usual anti-ivm talking points I've addressed in other threads, and I will assume that they have been added by the journalist. In this thread, I'll focus on the fraud claims, which are sourced to the investigations of the group.
Read 34 tweets
7 Oct
Let's start keeping track of Fauci calls for resignation, from mainstream voices, overt and covert, because I'm starting to notice a bit of a pattern. May be nothing, may be something. 🧵washingtonpost.com/opinions/2021/…
This was a bit of a subtweet, also (hint: Collins was there for 12 years, Fauci has been there for 37).
Then there was this shambles of an interview...
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!