Thread by @fperrywilson

Folks, we need to talk about this Vitamin D trial. I have no stake in this game - take Vitamin D if you want but this pre-print is super sus. (THREAD)
papers.ssrn.com/sol3/papers.cf…

The paper is presented as a randomized trial of vitamin D supplementation in hospitalized patients with COVID. Interesting and important question! And the results appear dramatic:

If true, this would be one of (if not THE most) effective treatments for COVID. But there are problems...

The first clue something is up is that the randomized groups aren't the same size:

It took me a while to figure out why this was, then I saw in the text that INDIVIDUALS were not randomized, WARDS in the hospital were.

OK - 8 wards, 5 randomized to Vitamin D, 3 to usual care. (Why not 4 and 4?? - but whatever). So this is actually a CLUSTER-randomized trial. That means you need to use CLUSTERED statistics to analyze it. They do not.

This is a big problem. But there is more. It seems that, even if the wards were randomized, the PATIENTS weren't randomized to the wards.

In other words, some hospital wards take different patients than others (different risk factors, etc). This is why we see this really weird finding in Table 1:

Baseline vitamin D levels dramatically lower in the "non-treated" group. Why? Preseumaby because different types of people got admitted to the wards than were randomized to usual care.

You'd expect people with low levels of Vitamin D to do worse - that has been shown multiple times - perhaps because higher Vitamin D levels are associated with less comorbidities.

Here's their Kaplan-Meier curve. It doesn't make sense. What do they mean by 'cumulative hazard' of mortality? What units are these? The overall mortality was 10% by their report.

I get frustrated with peer-review too, but this is why it's so important. This is super basic stuff - you don't call your study a randomized trial when it's a cluster randomized trial. And peer-reviewers would 100% have asked them to go back and redo the stats.

The authors could solve this, btw, by releasing a de-identified dataset (including the ward number) for this study. We could analyze it in about an hour at least for topline results using appropriate stats.

And again, is there harm from Vitamin D? Minimal honestly. The harm from promotion of studies like this is tweets like this that try to dissuade people from getting vaccinated and doing other protective measures.

https://twitter.com/Harrygrant123/status/1360886194180153344

So please, read skeptically. Pre-prints have been a boon in COVID times but this study is just... not well done. Be aware. (/END)

UPDATE:
Authors' post on PubPeer now says "we never say in the article that it is a randomized control trial (RCT) but we consider an open randomized trial, and an observational study."

Curiouser and curiouser.

pubpeer.com/publications/D…