Broken record here, but speaking as a scientist who deals primarily with strength/quality of statistical evidence, the crux for just about everything in science lies in philosophy.
Many, if not most statistical evidence failures come from ignoring it.
You don't need to read the complete works of 10k dead white guys, but it's incredibly valuable to dive down the "what does this even mean" rabbit holes.
Can't promise it'll make you more productive, but it will almost certainly make you a better analyst.
I am an amateur at sci phil, for what it's worth, but make sure to engage with those who know better to steer me in the right directions.
However, beware the "critical thinker" crowd. Often overconfident BS couched in pseudo sci phil. Hard to tell the difference.
(also, yes, know the quoted tweet is a joke, but did want to point out that STEM and phil are deeply connected)
(i have no idea whatsoever how philosophy progresses or what that even really means, but science definitely progresses more when scientists engage more with the phil that is at the root of everything we do)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
"Problems with Evidence Assessment in COVID-19 Health Policy Impact Evaluation (PEACHPIE): A systematic strength of methods review" is finally available as a pre-print!
One of the most important questions for policy right now is knowing how well past COVID-19 policies reduced the spread and impact of SARS-CoV-2 and COVID-19.
Unfortunately, estimating the causal impact of specific policies is always hard(tm), and way harder for COVID-19.
There are LOTS of ways that these things can go wrong. Last fall, we developed review guidance and a checklist for how to "sniff test" the designs of these kinds of studies. Check that out here:
A brief thread rant on woodworking and causal inference (yeah, you read that right).
From table legs to descriptive stats tables, from picture frames to the framing the big picture for studies. It's gonna get weird, but stick with me.
Let's say you want to make a very simple table. Easy! 4 legs cut to the same length and flat top. Step 1: cut those legs.
So, you take your leg material, and you carefully measure (twice) 26," mark it, and make your cut.
And no matter how careful you were, they don't match.
You might think that you didn't measure carefully enough, or cut straight enough. I promise that's not the problem.
The problem is that you were thinking about the problem the wrong way. Because unless you are a pro, measure twice cut once will NEVER get them to match.
The most important lesson I have learned throughout this disaster is this: there is no adult stepping into the room to fix things.
YOU are the one who is going to fix things if things are going to be fixed. No one else will.
I keep seeing my colleagues and friends doing amazing things making huge splashes all over. And while they are all wonderful brilliant people, the thing that makes them stand out is this one simple thing:
They didn't stop at "someone should do this." They just did it.
I have learned to no longer rely on the adults, whether they are well-respected people in my field, officials, family, etc. Some just failed to step up, some are getting in the way, and some worse.
I've lost a lot of heroes recently; I'm sure you have too.
In advance of the Danish mask study's expected publication, Sarah Wieten (@SarahWieten), Emily Smith (@DrEmilyRSmith), and I have written our concerns about its framing and design.
Our comments have been sent to DMJ editors, and are available on PubPeer.
1) The framing is atrocious, in particular since it implies epi-ists < econ-ists
2) The methods divide is very real, but not for the reasons implied.
3) Epi would, indeed, benefit greatly from embracing these methods, but
4) This article only hurts that effort.
The framing here is really bizarre. It starts with saying that RCTs exist and that other approaches can be used, but ONLY mentions the econ-preferred route.
Epi has an entire field of causal inference with observational data. To not even mention it is negligent.