Doing research on shrooms is hard, because they're mostly illegal and there are a lot of restrictions in most places
Much of the research is simply observational, asking people who already take shrooms what they think of the experience
So what did the researchers do?
They asked people who were already going to microdose shrooms to BLIND AND RANDOMIZE THEMSELVES
Super cool idea, and impressively sound methodology
Basically, they got people to put the drug and a placebo into opaque pills, then randomize the selection of the drug so that they didn't know what they were taking over the course of 4 weeks
Is this open to some bias? Yes, absolutely
Is it a genius idea that allows you to conduct really difficult research on illegal drugs cheaply and ethically? Also yes
Each participant had 4 weeks of randomly taking either a placebo or psilocybin, then recording their mood and mental health on a dozen questionnaires. A slightly modified randomized-crossover trial
And at the end of the study, the researchers found that there was no meaningful difference between people randomly allocated to take shrooms compared to those who had taken a placebo
Now, there are drawbacks to this methodology. Firstly, you can only run a study like this on people who were already planning on microdosing psilocybin and who are easy to find online, which it turns out is a somewhat specific group
There's also definitely potential for unblinding bias - as the researchers note, it's likely that some people checked to see which pills they were taking
But all in all, this is not only a really interesting study answering a useful question, it's also a research innovation that's going to be invaluable for the future of research into illegal drugs
• • •
Missing some Tweet in this thread? You can try to
force a refresh
In patients where a clinical cause was able to be identified, the vast majority of cases were clearly deaths caused by COVID-19, which means that this is likely a true undercount of the deaths
Worryingly, only 1 in 7 of the children who died of COVID-19 had been tested for it beforehand
Also, this was a beautiful thing to see while reading the study. "We made a mistake at the start so we fixed it but here's all the data so you can tell for yourself" is absolutely the right thing to do when reporting on your trial outcomes!
I suspect when I do a formal risk of bias score for the study it will come out looking fantastic simply from this one thing. Researchers who are entirely open about their methods are the ones who publish the best studies!
This appears to be a pretty straightforward case of p-hacking as described, except in the opposite direction - the authors kept hunting around until they found a NON significant relationship to put in the paper
Of course, the research letter that this is about, because it suggested that Swedish schoolchildren/teachers were totally safe from COVID-19, has been in dozens of news articles and has an Altmetric of 6,500 🙄
It sounds like the only reasonable course of action for @NEJM at this point is to place a notification of concern on the paper and investigate themselves
There's been a lot of discussion recently about preprints and how we should interpret them as evidence
A few thoughts from my perspective as an epidemiologist and science communicator 1/7
2/7 The basic idea of preprints was originally twofold:
1. Put your research in the public domain, get feedback before publication 2. Get research up while it works its way through months/years of peer review
3/7 During the pandemic, given the absurd reality of months+ for studies to be peer-reviewed, preprints have become one of the primary ways for us to see new science
Pretty amazing that, despite the spike in COVID-19 deaths from the Victorian outbreak, there was ~no~ excess mortality during winter because the number of deaths directly attributable to influenza fell from a yearly average of ~900 to 42
Also interesting - despite lockdowns and restrictions, at worst only very minor increases in deaths due to diabetes, cancer, or CVD, and a modest reduction in deaths due to COPD (probably flu related)
As an interesting note on this, the last time I contacted an author/editor to point out a study had mathematical errors, I was dismissed by the editor and insulted by the author
Conversely, I've had some really excellent discussions on Twitter about research, people are often happy to engage. Is this unprofessional?
Another important point to think about @apsmunro is that formal channels to critique research take at least months, and can easily stretch to years. The challenge with the status quo is that it often results in no action even for obvious mistakes