Thread by @vectorgen

I have some thoughts on the new eLife policy

https://twitter.com/cshperspectives/status/1583053060116733952

First, I applaud attempts to change the peer review landscape. It's broken. But this policy isn't it.

To call this "Peer Review without gatekeeping" is extremely disingenuous. It makes gatekeeping even worse

What is the point of peer review? Journals review for technical correctness and impact. Where they lay on the continuum varies between journals and their priorities

Behold! The Great Peer Review Matrix!

Any journal (or any publishing venue really) can be placed on this chart somewhere

Your high impact journals peer review for both impact and content. They're in the upper left

Journals such as PLoS ONE or PeerJ that review for technical correctness but not impact are in the bottom left quatrant (no judgements). Journals that start to fuzz the line (like a lot of the MDPI venues) start to move toward the right

Preprint servers such as bioRxiv that do not evaluate correctness or impact are in the bottom right quadrant

NOTHING SHOULD BE IN THE TOP RIGHT QUADRANT

This new eLife policy gives me pause. Maybe it could be technically consistent if every submitted paper was put through the system. But then it's just another preprint server and we have plenty of those already

Instead, the editorial board will decide which papers get through. That's not different from how it works everywhere. But the crucial difference here is that every paper selected by the board is "published"

Yes they make a distinction between "accepted" and "reviewed" but it all has that eLife shine and folk won't make the distinction, especially if all categories are indexed

But the editorial board can't realistically review all papers for technical correctness (they don't have the expertise in every field). So they're defacto reviewing for impact

So we have a situation where we have a semi-glam journal reviewing for impact, BUT NOT TECHNICAL CORRECTNESS. They're in the forbidden quadrant

eLife seems to be trying to have their cake and eat it too. They want the status that comes with being a glam journal, but want to be able to say "We don't act like other glam journals".

They're correct. They're worse.