Everyone should read this social science replication commentary. Extremely interesting. Note that it doesn’t actually replicate anything (yet). It just predicts what papers are suspect based on some basic indicators of quality. Still, there is much to learn from + say about this.
A few thoughts. Economics comes out looking relatively good, but I enjoyed this observation: “A unique weakness of economics is the frequent use of absurd instrumental variables.”
I like an IV once in a while but I agree 90% are junk and we should probably stop.
On the huge number of self evidently crap studies, I recall how Lant Pritchett once said that research is like ballet. Every moment there are millions of kids doing terrible ballet all over the world. But all that is necessary to product the best people & performances.
He didn’t mean elite researchers. He meant research that mattered. Like the economic policy in the middle of a massive crisis or the vaccine in a pandemic. Crap research is the sad production process along the way.
I don’t think that’s a totally correct way to feel sanguine about a lot of bad research though. We can collectively say no to n=23 studies with 12 outcomes and cherry picking the one chance result that is significant.
Author is @AlvaroDeMenard who seems like an unusually interesting Twitter follow as well
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.