This appears to be a pretty straightforward case of p-hacking as described, except in the opposite direction - the authors kept hunting around until they found a NON significant relationship to put in the paper
Of course, the research letter that this is about, because it suggested that Swedish schoolchildren/teachers were totally safe from COVID-19, has been in dozens of news articles and has an Altmetric of 6,500 🙄
It sounds like the only reasonable course of action for @NEJM at this point is to place a notification of concern on the paper and investigate themselves
I mean, seriously, based on that correspondence the lead author knew that schoolchildren had excess mortality of 68% (unless combined with preschool kids), but this is what they concluded
Oh, forgot to mention that this lead author was also an original signatory/author on the Great Barrington Declaration
Reading the emails is incredibly stark. I cannot see any avenue but retracting this paper asap
Just to update, I'm not sure calling for retraction was justified. It does not look good at all, but the authors have updated the paper with this information now
• • •
Missing some Tweet in this thread? You can try to
force a refresh
In patients where a clinical cause was able to be identified, the vast majority of cases were clearly deaths caused by COVID-19, which means that this is likely a true undercount of the deaths
Worryingly, only 1 in 7 of the children who died of COVID-19 had been tested for it beforehand
Also, this was a beautiful thing to see while reading the study. "We made a mistake at the start so we fixed it but here's all the data so you can tell for yourself" is absolutely the right thing to do when reporting on your trial outcomes!
I suspect when I do a formal risk of bias score for the study it will come out looking fantastic simply from this one thing. Researchers who are entirely open about their methods are the ones who publish the best studies!
There's been a lot of discussion recently about preprints and how we should interpret them as evidence
A few thoughts from my perspective as an epidemiologist and science communicator 1/7
2/7 The basic idea of preprints was originally twofold:
1. Put your research in the public domain, get feedback before publication 2. Get research up while it works its way through months/years of peer review
3/7 During the pandemic, given the absurd reality of months+ for studies to be peer-reviewed, preprints have become one of the primary ways for us to see new science
Pretty amazing that, despite the spike in COVID-19 deaths from the Victorian outbreak, there was ~no~ excess mortality during winter because the number of deaths directly attributable to influenza fell from a yearly average of ~900 to 42
Also interesting - despite lockdowns and restrictions, at worst only very minor increases in deaths due to diabetes, cancer, or CVD, and a modest reduction in deaths due to COPD (probably flu related)
As an interesting note on this, the last time I contacted an author/editor to point out a study had mathematical errors, I was dismissed by the editor and insulted by the author
Conversely, I've had some really excellent discussions on Twitter about research, people are often happy to engage. Is this unprofessional?
Another important point to think about @apsmunro is that formal channels to critique research take at least months, and can easily stretch to years. The challenge with the status quo is that it often results in no action even for obvious mistakes