Health Nerd Profile picture
27 Apr, 29 tweets, 9 min read
This is one of the larger trials of hydroxychloroquine, ivermectin, povidone-iodine and zinc for prevention of COVID-19

At first, it appears positive, but there are some worrying issues

Some peer-review on twitter 1/n
2/n The study is here, and it's a cluster-randomized controlled trial, where people living in dorms of Singapore were given one of the 4 treatments or a vitamin C control during large COVID-19 outbreaks in the dorms ijidonline.com/article/S1201-…
3/n The results seem to show that people who take HCQ or P-I have fewer infections than those who only have vitamin C, with a really impressive risk reduction
4/n If these reductions are true, either HCQ or P-I would be a massive, world-changing reduction in risk for the prevention of COVID-19 (although zinc and ivermectin would be largely useless)
5/n But reading the methodology, some things immediately spring out. Firstly, this was a cluster-randomized trial, where participants were randomized to the intervention by the floor of their dorm
6/n ...but this was NOT what the study was originally designed for. The sample sizes, statistical analysis, and other protocols were designed for a parallel-arm clinical trial

This is not ideal
7/ While cluster-randomized trials are not my forte, all of the statisticians I've asked so far have said that turning a 5-arm study into a cluster RCT in this way leaves you with quite a lot of issues
7.5/n In particular, by clustering at the floor level the researchers essentially reduced their effective sample size from 3,000 patients to 40 floors. You have to take this into account in the statistical analysis, which the study does not seem to have done
8/n There's also an issue with how COVID-19 was ascertained in the study. Patients gave a blood sample at day 1 and 42, and then medical records were searched to see if they'd had a positive PCR between those two dates
9/n So, firstly, that is an issue because there's a bias in the PCR testing - only those who had a test (for whatever reason) AND had it entered into whichever medical records system would show up as positive
10/n On top of that, the serology isn't great. Serology often misses people who are tested early in their infection, so the estimates of infection rates for all the groups are probably too low
11/n If you correct for this, instead of the percentages given in the results, you get much higher values for all the treatments, and a potentially reduced protective effect
12/n Also, the study excluded people who already had COVID-19 antibodies at the start, but there isn't much information about these people, which is an issue given that they were excluded after already being randomized (and treated?)
13/n Moreover, a full 20% of the sample was excluded because they were not from randomized floors, which is bizarre. How were they recruited, randomized, and apparently treated if they were not on randomized floors?
14/n It seems very much like this trial was conceived, planned, AND RUN as a randomized parallel-arm trial and then halfway through switched to a cluster RCT
15/n Ok, so that's all pretty worrying. The results may not be statistically viable, the methodology has quite a few flaws

But there's actually a potentially bigger issue
16/n The primary outcome reported in the trial was laboratory-confirmed (through either PCR or serology) COVID-19. This is the measure that the headline results is based on
17/n But if we go to the pre-registration for the study (which, incidentally, doesn't talk about controlling for a clustered design), there's something a bit weird

The primary outcome was originally "acute respiratory illness"
18/n So when the study was registered, in June 2020, the primary outcome was acute illness. A month after the final results came in, the primary outcome was changed to laboratory-confirmed COVID-19
19/n Moreover, the outcome that was registered in advance as the primary outcome has completely different results, showing a statistically significant effect for ivermectin but nothing else
20/n Now, there's nothing inherently wrong about switching outcomes, and the authors do mention a reason for changing it, but the fact that it was only changed after the study was finished is very strange
21/n The explanation in the text also doesn't quite make sense. The paper reports excluding people who had a positive serological test at baseline - how can these people have been tested if there were no serological tests when the study started?
22/n So what does this all mean?

Well, overall, it's quite hard to trust the trial's results
23/n The study does not appear to follow the guidelines for implementation and analysis for cluster RCTs, which means that it's hard to know what to make of the analysis ncbi.nlm.nih.gov/pmc/articles/P…
24/n The primary outcome was also switched, with a bunch of other odd inconsistencies in the research that make it a bit hard to know if the conclusions hold water
25/n To their credit, the authors talk about some of these things in the limitations section of the study, but not all of them and I'm not sure they really explain why these are not issues
26/n Anyway, I'm not sure I would rely on this study as evidence for much, despite the large size
27/n Apologies, one of the above tweets is wrong. The authors did indeed take into account the clustering in their statistical analysis
28/n The more I read this paper, the weirder it sounds. So they randomized ineligible people (how?) from two floors that were not clustered, and then assigned them to vitamin C if the other medications were contraindicated?

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Health Nerd

Health Nerd Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @GidMK

24 Apr
The weirdest thing about the whole herd immunity through natural infection argument is that it's never happened ever for any disease long-term so it was always a wild idea for COVID-19
Like, sure, pandemics died out - eventually most diseases became endemic and killed only a small number of people each year

But that's definitely what's been bandied about as herd immunity
Imagine if instead of "herd immunity" the message had been "recurring outbreaks with a slowly diminishing fatality rate until after months/years the number of yearly deaths would get low enough to not bother any more"
Read 6 tweets
22 Apr
There's a silly non-COVID-19 science headline I'M SO HAPPY GUYS
The study itself is interesting - sleep duration and risk of dementia, lots of follow-up, decent sized sample (although relatively few events) nature.com/articles/s4146…
But the headline is super misleading for so many reasons. My faves:

1. absolute risk is really small (~1 case per 1,000 person-years)
2. The authors acknowledge later in the article that they don't know if this is causal or not
Read 5 tweets
21 Apr
2021 will hopefully be the year that the armchair epidemiologists stop being wrong about infectious disease, excess mortality, etc, and move on to being wrong about something else

Maybe economics?
To clarify, because of course I need to (sigh) this is a joke about the twitter randoms who have deemed themselves experts not a critique of interdisciplinary work
I'm currently working on a paper with 3 economists, an immunologist, a demographer, and 2 statisticians on COVID-19. Non-epis have great and valuable insight!
Read 4 tweets
20 Apr
One interesting point is that this article gets several facts wrong. Whether that detracts from the commentary on science or not is I suppose up to the reader
This statement, for example, isn't really true. The U.S. has had school closures much less severe than (for example) South Korea, or a dozen other places. The reference only talks about Europe!
"Studies have repeatedly concluded" - links to a tweet, and two articles on teacher's unions. There are many studies that have concluded precisely the opposite
Read 5 tweets
20 Apr
"Chronic disease has caused COVID-19 deaths, if we didn't have so much diabetes fewer people would've died" - incredibly dumb argument for many reasons, not least that it is true of LITERALLY ALL HUMAN DEATHS
Yes, if we had solved the biggest medical issue of the modern age fewer people would've died of COVID-19

What of it?
I mean, seriously, we've been trying to 'fix' NCDs for decades, and while they are in theory somewhat preventable they are still a large and growing problem in most places in the world
Read 4 tweets
19 Apr
Q: How many people have died globally from COVID-19?

A: Probably more than the confirmed count, but it's really hard to say

coronavirus.medium.com/how-many-peopl…
The basic idea here is that we could be either undercounting or overcounting COVID-19 deaths

I think the most likely explanation is some combination of the two
Based on some very careful examinations of death reporting systems, we can say that there are probably some portion of deaths that are recorded as due to COVID-19 but were not caused by the virus
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!