Also, this was a beautiful thing to see while reading the study. "We made a mistake at the start so we fixed it but here's all the data so you can tell for yourself" is absolutely the right thing to do when reporting on your trial outcomes!
I suspect when I do a formal risk of bias score for the study it will come out looking fantastic simply from this one thing. Researchers who are entirely open about their methods are the ones who publish the best studies!
Sorry, wrong screenshot above. This is where they are honest about the research issues
Seriously, there are so many studies that just exclude data like this and never report mistakes, or do so in a thoroughly opaque way. This is a breath of fresh air, and makes me trust the study far more
• • •
Missing some Tweet in this thread? You can try to
force a refresh
In patients where a clinical cause was able to be identified, the vast majority of cases were clearly deaths caused by COVID-19, which means that this is likely a true undercount of the deaths
Worryingly, only 1 in 7 of the children who died of COVID-19 had been tested for it beforehand
This appears to be a pretty straightforward case of p-hacking as described, except in the opposite direction - the authors kept hunting around until they found a NON significant relationship to put in the paper
Of course, the research letter that this is about, because it suggested that Swedish schoolchildren/teachers were totally safe from COVID-19, has been in dozens of news articles and has an Altmetric of 6,500 🙄
It sounds like the only reasonable course of action for @NEJM at this point is to place a notification of concern on the paper and investigate themselves
There's been a lot of discussion recently about preprints and how we should interpret them as evidence
A few thoughts from my perspective as an epidemiologist and science communicator 1/7
2/7 The basic idea of preprints was originally twofold:
1. Put your research in the public domain, get feedback before publication 2. Get research up while it works its way through months/years of peer review
3/7 During the pandemic, given the absurd reality of months+ for studies to be peer-reviewed, preprints have become one of the primary ways for us to see new science
Pretty amazing that, despite the spike in COVID-19 deaths from the Victorian outbreak, there was ~no~ excess mortality during winter because the number of deaths directly attributable to influenza fell from a yearly average of ~900 to 42
Also interesting - despite lockdowns and restrictions, at worst only very minor increases in deaths due to diabetes, cancer, or CVD, and a modest reduction in deaths due to COPD (probably flu related)
As an interesting note on this, the last time I contacted an author/editor to point out a study had mathematical errors, I was dismissed by the editor and insulted by the author
Conversely, I've had some really excellent discussions on Twitter about research, people are often happy to engage. Is this unprofessional?
Another important point to think about @apsmunro is that formal channels to critique research take at least months, and can easily stretch to years. The challenge with the status quo is that it often results in no action even for obvious mistakes