Maybe it's because lots of paper aren't peer reviewed, maybe it's because diff fields diff expectations, but there is something deeply disturbing about scientific research on Covid.
Most papers that use statistical methods are deeply flawed and not even in a sophisticated way
Most papers that use statistical methods are deeply flawed and not even in a sophisticated way
I am interested by Twitter discussions on masks, lockdowns, airborne transmission, HCQ and they often end by someone tweeting a link to some publication - supposedly the final argument.
But depressingly it very often takes less than 5 minutes to find a major flaw
But depressingly it very often takes less than 5 minutes to find a major flaw
A few examples.
A piece on the impact of the french local elections on Covid in France was widely publicized. As @Emmanuel_LEVY wrote, the stats are total nonsense.
A piece on the impact of the french local elections on Covid in France was widely publicized. As @Emmanuel_LEVY wrote, the stats are total nonsense.
I read a piece claiming masks were vital to stop the pandemic. How do they know? They use a SEIR model and assume that the transmission rate is monotonous with the share of masks in the population.
I mean if you start by assuming the conclusion, it won't be hard to prove.
I mean if you start by assuming the conclusion, it won't be hard to prove.
A piece (quoted by that famous JPM study) explained that lockdown were useless because looking at various countries with various lockdown policies, many countries had a decreasing trend and "the shapes of the curves looked similar"
Seriously???
Seriously???
Let's not even discuss that HCQ study in the Lancet ; it has so many red flags it should be used for bull fighting.
I could go on... and honestly this puzzles me a lot. I don't know the field well enough to know what's the deep reason for this. Pressure to publish? Financial incentives? Not enough stats in college? Difficulty to reproduce? Any insights welcome !
And to clarify : I have also read some studies I have found very good. I'm not over generalizing here. But on average it is extremely disappointing.
Also I not a tendency to use very sophisticated methods with fancy names...but with 20 or 30 data points. Not robust at all.
Also I not a tendency to use very sophisticated methods with fancy names...but with 20 or 30 data points. Not robust at all.
If you have less than hundreds of data, mispecification of the model is a much bigger risk than any confidence interval you will calculate- and you end up with false precision risk. Something I find extremely rarely discussed in those papers.
I notice a tendency*
