1/ Why I am a deep peer review skeptic, even though I'm a journal editor and my articles (sometimes!) make it through peer reviews.
2/ Background: @khberglund drew my attention to this article that reviews problems with peer review. And they are many! Let me tell you about them... elemental.medium.com/amp/p/1d58e5d9…
3/ Perhaps the most fundamental problem is that peer review is not reliable in the sense that peer reviews are weakly correlated with each other. A famous study by Peters and Ceci is a great example...
4/ Peters and Ceci submitted 12 *already* published articles to psych journals. Of the 9 that were not detected by editors, 8 were rejected by the journal that previously published it!! cambridge.org/core/journals/…
5/ Another problem is that peer review wasn't terribly good at weeding out major scientific problems. Humphries correctly notes that peer review did not prevent the replication problem in the social sciences, which is simply gigantic.
6/ I would also add another problem that peer review seems to make worse: p-hacking. In fact, there's an argument to be made that much bad quant practice is made worse by peer review.
7/ Humphries' is also correct in that peer review doesn't really change the paper. People just revise the same argument until some journal accepts it.
6/ One might argue that peer review is like democracy: it sucks, but the alternatives are worse. Not sure about that. Humphries' correctly notes that much great science - relativity theory, Darwinism, etc - was not peer reviewed.
9/ As an editor, I still use peer review, but I greatly curtail it. I ask for short reviews, I form my own judgment, I rarely do R&Rs.
10/ Bottom line: We have had about 50 years of the current peer review journal system and I am not impressed.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
