Thread by @GidMK

This vitamin D/COVID-19 study has gone viral, because the results appear to be impressive and people love promoting vitamin D

Unfortunately, the study itself is...problematic

Some peer-review on twitter 1/n

https://twitter.com/DavidDavisMP/status/1360647462197878791

2/n The study itself is here papers.ssrn.com/sol3/papers.cf…

At first glance, it appears to be a randomized controlled trial comparing calcifediol (vitamin D metabolite) to a control for severe COVID-19 with amazing outcomes (60% mortality reduction 👀)

3/n So, if you only glance at the abstract, you get the picture of an amazing positive result for vitamin D

But reading further, the problems start almost immediately

4/n Firstly, the study type. People are talking about this as "randomized" because the authors use that word in the abstract

But the authors didn't actually randomize patients!

5/n What the authors describe doing is, depending on how you read it, either a cluster-randomized trial with a sample size of 8, or a completely uncontrolled observational trial

I think it's almost certainly the latter

6/n Why? Well, a big clue is that this is what the authors got ethics approval to do. Another clue is that nowhere in the study is an actual method for randomization described!

7/n If the authors did indeed do a randomized trial on participants who consented only to a cohort study, it is a decidedly non-trivial issue

Where I live, there would be firings, lawsuits, and potentially criminal proceedings

8/n More likely, in my opinion, is that this is simply as the authors describe an observational cohort study of people in hospital who were either given calcifediol or not. They just use the word randomize incorrectly

9/n But even then, there are massive issues

For example, the PRIMARY ANALYSIS (60% mortality reduction) excludes ~20% of their total sample because of missing baseline data on vitamin D status

10/n And we get no information whatsoever on these missing people. Were they from the treated group? The control? Did they die, go to ICU etc? We have no idea!

11/n Another issue is that there were 8 presumably quite different COVID-19 wards, but the authors basically ignore these differences. There is no discussion of the purpose of the wards, and no correction for it in the statistical model

12/n Could the results be explained by different wards having different admissions protocols? Potentially, but we are given no information to make this assessment at all

13/n This is even more troubling when you consider that the baseline vitamin D levels are different in the treated and control patients (for whom there was vit D information)

So we know that the wards were different, but we don't know how much or why

14/n There's more. If this WAS indeed a cluster RCT (I'm skeptical), then the authors did the wrong analysis and the results are probably actually non-significant

Excellent thread here:

https://twitter.com/lycraolaoghaire/status/1360765704849481731?s=20

15/n There are also numerous issues I haven't covered that @fperrywilson does a great job dissecting here:

https://twitter.com/fperrywilson/status/1360944814271979523?s=20

16/n There are other odd inconsistencies with the study. For example, they had 93 patients die and 110 ICU admissions

That's a startlingly high ICU mortality rate, unless a majority of patients who died were not admitted to ICU

17/n For some context, the average ICU mortality was about 30% in Spain at the time, which is still really high but not "almost 100%" high

sciencedirect.com/science/articl…

18/n The timelines are really weird. The study is reported as going for 91 days (1st March-31st May) but the K-M curves have 100 days for the treatment group. Probably a minor error, but still strange

19/n The K-M curves are also just bizarre. My guess is that they've confused the outcome death with the inverse for the second graph, but even then they really make little sense

20/n Also, the study ran until May 31st. This magnificent, amazing, literally earth-shattering result (60% REDUCTION IN MORTALITY) took...8 months? And no news, barely any press, for a treatment that could save 60% of people with COVID-19?

21/n The study also wasn't pre-registered, which is an issue considering it started on March 1st 2020. When was the treatment protocol (HCQ+azithro and dexamethasone) decided on?

22/n Also, how in the world did one relatively modest-sized hospital have 8 fully-dedicated COVID-19 wards open in Barcelona, at a time when Spain itself only had a handful of COVID-19 cases? Any Barcelonian followers who can elaborate?

23/n The same author group wrote another paper on the same patient population in January. Why was there no mention of this trial, or the MASSIVE mortality reduction anywhere? None of this is necessarily disqualifying, it's just really odd!

24/n I should specify that none of this makes the study totally sketchy, it's just all really weird and they are things that the authors should have explained in the paper

25/n Overall, what we have is a study that, if run as specified, was a non-randomized prospective cohort study that gives us very little/no new information on vitamin D for COVID-19

26/n This doesn't mean that you shouldn't take vitamin D, it's relatively low-cost and the harm is mostly to your wallet, but the jury is still out whether it will have any benefit at all to COVID-19 🤷‍♂️

27/n Update: the authors have confirmed that this was not a randomized trial on PubPeer. They are still using the word "random" in a very confusing way, but what is described here is not an RCT by any description

28/n And I wrote a piece about this trial and COVID-19 and vitamin D in general gidmk.medium.com/vitamin-d-and-…