This chart—demonstrating the causes of the replication crisis back in 1975—is important.

I’d like to offer a slightly different interpretation of the phenomenon than is typical… 🧵

Why does this happen? Explanation #1: scientists don’t understand statistics. Definitely true, but doesn’t explain the magnitude or directionality of the effect, I think, and efforts to correct it don’t seem to help much. Stats are hard but scientists aren’t that dumb…
Explanation #2: distorted career incentives to publish “positive” results lead scientists, consciously or unconsciously, into misuse of methods (garden of forking paths, etc.)

Definitely true, but who is setting those incentives and why? Mostly other scientists…
Obviously, everyone wants positive results because they are meaningful and interesting, and failure to disconfirm the null hypothesis is… actual failure.

This is heresy! But I want to explain and agree with scientists’ revealed preferences here.
Rationalist philosophy of science says that science seeks “knowledge,” which it takes to mean “true facts.”

If that were true, for any proposition p, we should be equally happy to find evidence for p and not-p. Both are knowledge! Publish “negative” results, they are progress!
Everyone’s revealed de facto belief is that this is not what science is about.

If true, proposition p would be exciting and worth publishing; if false, proposition not-p is inconsequential.
Science seeks understanding, not knowledge.

That is: better ontology, not incremental epistemological additions.

Facts have little value unless they contribute to insight. Uninteresting theories are uninteresting whether true or false.
The replication crisis is partly due to a mismatch between institutional science norms created on the basis of a mistaken philosophy of knowledge, conflicting with our natural and accurate sense of science as a quest for understanding.
(Uh, that was Explanation #3, I had slightly lost the thread of the framing here.)
Recognizing the mismatch does not imply rejecting norms for establishing knowledge. Knowledge is often important!

p-hacking, selective publication, etc, are major problems that need fixing.

I’m suggesting that they aren’t accidental problems. The root cause is elsewhere.
A more accurate understanding of what science is, what it is for, and how we do it may lead to reforms of norms that address the problems at a deeper level.
We should acknowledge that there are some things we hope are true, because they would be interesting. Norms that assume we are equally happy with the null hypothesis encourage flouting and gaming of the norms.
Starting by acknowledging that we seek interesting understandings, we can ask what scientific norms would promote that goal.

I suspect these would be quite different from our existing ones (which were invented mid-20th-century based on then-current wrong philosophy of science).
This does not imply rejecting or neglecting the existing norms; rather, improving their operation by putting them in proper context.
Negative results absolutely should be made public—and we should also admit they’re disappointing.

The dual functions of publication as information dissemination and accreditation was pernicious, but a limitation of paper tech. Availability and interestingness can now separate.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with David Chapman

David Chapman Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @Meaningness

26 Apr
Whoa! So De Gandillac, who supervised the PhDs of all the significant pomo pioneers, was concerned with the preeminent value of technological progress, as advocated by Nicholas of Cusa (who I knew only as a the name of some vague Medieval theologian)…
Now imagining de Gandillac reading Derrida's _Of Grammatology_ and thinking "Oh god, what did I do to deserve this, another pomo thesis, my field is Medieval philosophy of technology but somehow I am personally responsible for the collapse of Western civilization"
Why had I heard of Nicholas of Cusa?

Figured out: he’s discussed repeatedly in Thomas Kuhn’s _The Copernican Revolution_ as one of Copernicus’ inspirations.

(This book is much less well-known than his _Scientific Revolutions_, but it is excellent and should be more widely read)
Read 4 tweets
24 Apr
🧛🏻‍♀️ Last night I finished writing the first new chapter in a decade of my supposedly-web-serial tantric Buddhist vampire romance novel.

🧛🏻‍♀️🥀🍷 I will publish this first love scene of the novel on May Day, next Saturday, for the date's traditional associations en.wikipedia.org/wiki/May_Day Image
🧛🏻‍♀️📕 The new chapter of my vampire novel won't make much sense out of context, so if you think you might want to read it, you could try reading the book from the beginning now.

buddhism-for-vampires.com/the-vetalis-gi…
Read 7 tweets
23 Apr
I know almost nothing about Stoicism. Advocates: what book or long article makes the stronghold case that it has substantive and significant content?

(From a distance, it appears to be a hope that something with desired properties must exist, with no demonstration that it does.)
I infer this from observing that criticism of Stoicism is usually met with No-True-Scottsman-ing: “That’s an ignorant misunderstanding of Stoicism; the real thing is totally different.” What is this real thing?
This feels similar to the pattern around Critical Rationalism. When anyone says “there’s no there there, you haven’t got a thing,” there’s a chorus of “You don’t understand, it’s totally the answer to everything, there’s a conspiracy to deny its awesomeness.”
Read 6 tweets
18 Apr
In order to do the things everyone does, you have to say the things everyone says. Unless you are willing to be a weirdo—which may exclude you from doing things anyway.
This explains the otherwise puzzling fact that the only part of the otherwise dead trad MSM anyone is willing to pay for is culture war opinions. Saying the correct thing about this morning’s outrageous noodle incident is genuinely worth paying $thousands/year for social cred.
Someone pointed out recently that what you get from most of the top-by-revenue substacks is access to correct culture war opinions a few hours ahead of the mob. Pay for them so you can sound superior. (The others are all financial advice, which also has obvious dollar value.)
Read 8 tweets
15 Apr
Why do *I* always get stuck with the thankless, grindy tasks the official experts in the field evade responsibility for?

Professional historians of science managed to gloss over “why exactly did logical positivism fail” because it was hard, so I had to write that up. Now this.
Historians’ passing over of the death of logical positivism: the camera tastefully cuts away from the scene and all we hear are muffled screams in another room. tvtropes.org/pmwiki/pmwiki.…
They had to kill everyone off before starting the sequel (“Postmodernism Does Science”), but they’d written themselves into a corner.

Coming up with a convincing story for how the heroes of Season One all suddenly died would take way too much exposition. tvtropes.org/pmwiki/pmwiki.…
Read 5 tweets
15 Apr
Research reveals that there are two kinds of guides to writing sex scenes. The first advises you, in perfectly seriousness, to include all the most ghastly clichés (without characterizing them as such). The second goes through the same list and says “these are ghastly clichés.”
100% of the fiction writing guides I have read, I came away thinking “if you need this advice, you absolutely should not consider writing fiction even as a hobby.”
I read somewhere “sex scenes are highly technical; do not attempt without special training.” Because I am an idiot, I got intimidated by that, and somehow imagined that books about how to write properly literary sex scenes would say something non-obvious.
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!