YES!!!

Most days in metascience, it feels like the odds are impossible, it's hard to believe that we'll ever make any progress at all.

And then every so often, something great happens.

This is a big deal for the future of science and publication, and I am STOKED!
Full disclosure: I contribute every so often to the NCRC team under the fantastic leadership of @KateGrabowski and many others, and have been a fan of both NCRC and eLife since they started (well before I started helping).
At some point I'll do a long thread about why this small thing is a WAY bigger deal than it sounds, but to tease: this heralds active exploration of a fundamental and long overdue rethinking and reorganizing of how science is assessed and distributed.
It's also the kind of experiment helps pave the way to conditions under which that change is possible, and an actual viable path for quality assessment and review in the open age.

It's a potential paradigm shift, but in the real way and not the lame windows 95 way.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Noah Haber

Noah Haber Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @NoahHaber

14 Feb
Folks: There are serious statistical, design, language, and ethics concerns with that vitamin-D RCT.

AT BEST, it's completely meaningless due to negligent statistical treatment and design, but there's more "questions"

Help us out: avoid sharing until the critics have had time.
Seriously, we (people who are frankly pretty good at this) are having a very hard time figuring out what the hell is happening in that trial.

Please give us time to do our work.
This thread is a good place to start if you want a taste.

But "super sus" is right; there is just so much here that doesn't make any sense at all, and this thread only scratches the surface.

It's gonna be a while before we figure this out.

Read 36 tweets
25 Jan
"Problems with Evidence Assessment in COVID-19 Health Policy Impact Evaluation (PEACHPIE): A systematic strength of methods review" is finally available as a pre-print!

doi.org/10.1101/2021.0…

THREAD!
One of the most important questions for policy right now is knowing how well past COVID-19 policies reduced the spread and impact of SARS-CoV-2 and COVID-19.

Unfortunately, estimating the causal impact of specific policies is always hard(tm), and way harder for COVID-19.
There are LOTS of ways that these things can go wrong. Last fall, we developed review guidance and a checklist for how to "sniff test" the designs of these kinds of studies. Check that out here:

arxiv.org/abs/2009.01940
Read 30 tweets
23 Jan
Broken record here, but speaking as a scientist who deals primarily with strength/quality of statistical evidence, the crux for just about everything in science lies in philosophy.

Many, if not most statistical evidence failures come from ignoring it.
You don't need to read the complete works of 10k dead white guys, but it's incredibly valuable to dive down the "what does this even mean" rabbit holes.

Can't promise it'll make you more productive, but it will almost certainly make you a better analyst.
I am an amateur at sci phil, for what it's worth, but make sure to engage with those who know better to steer me in the right directions.

However, beware the "critical thinker" crowd. Often overconfident BS couched in pseudo sci phil. Hard to tell the difference.
Read 5 tweets
11 Oct 20
"Measure twice, cut once" is bullshit.

A brief thread rant on woodworking and causal inference (yeah, you read that right).

From table legs to descriptive stats tables, from picture frames to the framing the big picture for studies. It's gonna get weird, but stick with me.
Let's say you want to make a very simple table. Easy! 4 legs cut to the same length and flat top. Step 1: cut those legs.

So, you take your leg material, and you carefully measure (twice) 26," mark it, and make your cut.

And no matter how careful you were, they don't match.
You might think that you didn't measure carefully enough, or cut straight enough. I promise that's not the problem.

The problem is that you were thinking about the problem the wrong way. Because unless you are a pro, measure twice cut once will NEVER get them to match.
Read 17 tweets
9 Oct 20
The most important lesson I have learned throughout this disaster is this: there is no adult stepping into the room to fix things.

YOU are the one who is going to fix things if things are going to be fixed. No one else will.
I keep seeing my colleagues and friends doing amazing things making huge splashes all over. And while they are all wonderful brilliant people, the thing that makes them stand out is this one simple thing:

They didn't stop at "someone should do this." They just did it.
I have learned to no longer rely on the adults, whether they are well-respected people in my field, officials, family, etc. Some just failed to step up, some are getting in the way, and some worse.

I've lost a lot of heroes recently; I'm sure you have too.
Read 8 tweets
12 Sep 20
STATS QUIZ!

I have the datapoints below. Nothing hidden, no tricks, just a bunch of data making roughly an ellipse.

In your head, draw what you think the ordinary least squares line (i.e. good ol' y= mx+b) line looks like for these data.
Is this what you drew?
Seems "obvious" right?

Except that's not the OLS line.

The red dashed line is the OLS line.

What's going on here?
Read 13 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!