My Authors
Read all threads
THREAD: Two bits of excellent #COVID19 news today. First, remdesivir significantly shortened disease in a 800-patient randomized placebo-controlled trial run by NIAID. Details will come later. Even more interesting though... (1/n)
statnews.com/2020/04/29/gil…
Remdesivir also came very close to showing significantly shortened disease in the randomized control trial from Hubei, China, of 237 severe patients (18 vs 23 days). Detailed data were published today in the Lancet. Why is this more interesting? (2/n)
thelancet.com/journals/lance…
The Hubei data are interesting because they show an effect that is (in my deductive scientific opinion) clear but that by the rules does not exist. How can I claim something is scientifically clear when it does not exist? The answer has to do with intuition vs sensing... (3/n)
Clinical improvement rates in the Hubei-Lancet study for all 237 patients are shown in the Kaplan-Meier chart below. Remdesivir vs placebo had a hazard ratio for clinical improvement of 1.27 with 95% confidence interval 0.89 to 1.80. What the heck does this mean? (4/n)
The curve shows the fraction of patients improved on 2 points in a severity scale or discharged from the hospital over time. The hazard ratio (poor term, really a benefit ratio) of 1.27 means you are 1.27x more likely to improve on remdesivir. So what's not to like? (5/n)
What's not to like is the 95% confidence interval, the range of hazard ratios accounting for 95% of statistically possible outcomes, is 0.89 to 1.80. This number overlaps 1, which would be no benefit. This makes the trial officially a failure. But is it really? (6/n)
A strict statistician would say we can't make any conclusions from the trial, it was a failure and the 1.27x number means nothing. I say not so fast; the trends can mean something. First, we must remind ourselves why we require effects to be 95% statistically likely... (7/n)
The 95% convention means we accept that 5% of conclusions from statistics studies will actually be wrong. The field has decided not to go lower to say 90% because having 10% of conclusions needing future retraction seems too high. However we should remember that... (8/n)
A benefit ratio of 1.27 with a 95% confidence interval of 0.89 to 1.80 still means more likely than not there is a real benefit, and the best estimate of that benefit is 1.27x. In addition, the benefit ratio improves for patients who start treatment earlier (graph)... (9/n)
This is as expected for an antiviral medication, so by fitting expectations is somewhat reassuring. I am not saying we can call this trial a success, but it's also scientifically and logically invalid to call it a complete failure. Why do I bother going through this? (10/n)
The Hubei-Lancet study would be a moot point if the NIAID study does indeed meet its endpoint with >95% statistical confidence. I bring this up because clinical trial near-misses are almost always described as complete failures, which is confusing and can be harmful... (11/n)
The NY Times for example said today that the Hubei-Lancet trial "reported that the drug offered no benefit to severely ill patients in China". NOOO!!! The trial did not report that. It reported that the benefit was not statistically significant... (12/n)
nytimes.com/2020/04/29/hea…
This is a common error of description by medical journalists (and also by some doctors) but we MUST stop making it. It creates the misimpression that there are contradictory trial results, which causes people confusion and distrust of actual positive results... (13/n)
Conflating near-misses of statistical significance with no effect is failing to realize that statistics is about degrees of certainty, not about black and white. This confusion can also have the effect of creating false equivalences between near misses and direct hits... (14/n)
Because statistics is about interpreting uncertainty, it helps to bring other scientific approaches such as an understanding of hypothesis and mechanism to interpretation of statistical results. Yes it's messy but the world is messy... (15/n)
BTW when WHO leaked the Hubei trial top-line results, Gilead said "'trends in the data suggest a potential benefit for remdesivir, particularly among patients treated early in disease" which appears correct (16/n) statnews.com/2020/04/23/dat…
Finally it is worth noting the trial meant to enroll 400 patients but only got 237. With 400, the benefit may have been statistically significant (confidence intervals shrink with more samples). Thus one could say the trial failed the drug, not the drug failed the trial. (n/n)
Missing some Tweet in this thread? You can try to force a refresh.

Enjoying this thread?

Keep Current with Michael Lin, PhD-MD

Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!