Thread by @iowahawkblog

Update on the bizarre Surgisphere COVID data base saga
theguardian.com/world/2020/jun…

Before reflexively hitting the "like" on this, take 5-10 minutes to read the entire article first. This is too important a story to engage in mindless headline-think.

I guess I should just accept that 90% of my timeline is either all-in on "HCQ is the miracle cure that the globalists don't want you to have" or "LOL HCQ is fish tank rat poison for Trumptards"

Yeah, a number of people shared various news reports on the University of Minnesota HCQ study with me yesterday. I have thoughts.

https://twitter.com/ipswichdude/status/1268553052992278528

Rather than link to the news stories about it, here is the actual NEJM published article. Go ahead and read it. It won't bite.
nejm.org/doi/full/10.10…

But since everybody seems to consume their news in bite size chunks that conform to their priors, here's the upshot:

The U Minnesota researchers recruited 821 participants, 719 of whom were recently exposed to someone with CV19.

/1

The goal was to see if HCQ had any impact on whether participants experienced CV19 symptoms following likely exposure.

The participants were randomly assigned to receive either HQC or a vitamin placebo.

/2

I will interject to say randomized trials are a very very good thing, the gold standard in experimental design, and there is no reason to suspect hanky panky in this.

/3

Here are the results: among those who received HQC, 11.8% eventually experienced COVID symptoms; among those in the placebo control group, 14.3%.

The HQC group saw a small, but not statistically significant improvement over the control group.

/4

I would note that once this study was curated for you by J-school grads, it was generally interpreted as proof that HQC has no effect on COVID.

/5

Now let's have a short talk about hypothesis testing, statistical significance, and Type I vs Type II errors.

In studies like these there is a null hypothesis, and an alternative hypothesis. In this study the null hypothesis was that HCQ has no impact on symptoms.

/6

The alternative hypothesis was that impact could be either positive or negative; that's known as a 2-tailed test.

The slight (2.4%) mean improvement for HCQ group was not enough to reject the null hypothesis at 95% confidence level, which is a standard for these thing.

/7

Note the "p=0.35" thing in the article abstract. This is the probability of a Type I error, concluding the null hypothesis (no impact of HCQ) is wrong when it is actual correct.

In short, a small improvement, but there is a 35% prob that could have resulted by chance.

/8

At this point I would note that significance tests are a function of sample size; here it was relatively low (400 or so in each group). If this same 11.8% vs 14.3% difference had been seen in a sample of 8000 patients, it would have been a slam dunk finding in favor of HCQ.

/9

Before you HCQ stans start screaming A-HA, that hypothetical study result of 8000 patients is *hypothetical*. This actual one had 800, and you work with the sample size you have.

/10

Now, there's also the issue of Type II error: failing to reject the null hypothesis (HCQ has no impact) when it's actually false.

Without belaboring this, the results of the UM study also had a high probability of Type II error. It in no way "proved" that HCQ had no impact.

The real result of the UM study (again, a well designed study):

If you think HCQ has no impact on COVID symptoms, or if you think it has a small but positive impact on COVID symptoms, these results aren't strong enough to conclude you're wrong.

/12

On the other hand, if you think that HCQ is The 'Rona Miracle Medicine, or if you think HCQ is Dr Drumph's Death In a Bottle, the Minnesota study provides solid statistical evidence that you're a complete idiot.

/fin