,
34 tweets,
8 min read
Read on Twitter
Since there's been a lot of discussion today about @eLife prompted by @TanentzapfLab, I thought it would be a good time to discuss several initiatives we're taking to reshape peer review.
I'll say at the outset - as I've said many times before - I think journals are an anachronism - a product of the historical accident that the printing press was invented before the Internet. I want to get rid of them.
More specifically, I want to get rid of pre-publication peer-review and the whole "submit - review - accept/reject - repeat" paradigm through which we evaluate works of science and the scientists who produced them. This system is bad for science and bad for scientists.
Evaluating works at a single fixed point in time, and recording the results of this evaluation in a journal title is ab absurd and ineffective way to evaluate science - it's also slow, expensive, biased, unfair and distorting.
A clear alternative is emerging - we should let scientists publish their work when they want to on @biorxivpreprint or the equivalent, and then review and curate these works in many different ways, throughout the useful lifetime of a paper.
In this future world we'll still have peer review - and maybe things that look a bit like journals to organize it - but it will differ in many crucial ways:
1) It won't be exclusive -- anyone, or any group, should be able to review a paper, so long as they do so constructively. It's silly to leave something as a important as evaluating the validity and contributions of a work based on the judgment of 3 people
2) It won't just happen once -- there's value in assessing a paper at the time it's published, but also in looking at it a year, 5 years, 100 years later.
3) We won't record the results of peer review in journal titles -- it's really crazy that we reduce something as multidimensional as the assessment of a work of science to a single yes/no decision (especially given how poisonous this system has become).
So you might rightly ask why did I agree to take over as EIC @eLife given that, despite all the good things it has done since its launch, is still at the end of a day a journal that does all of the things I just said we shouldn't do.
The reason is that it's all fine and good to have these ideas, but it's just noise if we can't actually implement such a system and get people to publish in it.
There are a ton of questions about how to turn these ideas into a functioning system that actually IS better for science and scientists, and actually fixes the problems of the current system.
And scientists are - completely understandably - very conservative about publishing. They may want to change things, but they have to get jobs, grants, etc... And right now this requires publishing in traditional journals. It shouldn't, but it does. That's reality.
So I was very excited when I pitched all of these ideas to the @eLife board, and they expressed broad support for this vision and hired me as EIC to try to make it work.
The opportunity in my eyes was to, with the backing of @eLife's funders and supporters, and with the amazing and dedicated editors, reviewers, authors and staff who built eLife, use it as a foundation for experimenting with and implementing this new model.
It's not a trivial undertaking -- it wouldn't make any sense to just come in and rebuild @eLife from scratch. Instead we are launching initiatives that will move eLife from where it is now to this future while keeping (and expanding as we can) our flow of excellent papers.
Towards this end, we're are taking two important steps in the next few months. They're both designed to take advantage of the fact that eLife has a lot of papers and our editors and reviews put a lot of work into evaluating hundreds of papers a month.
Initiative 1: Publish, for accepted papers, a statement from editors explaining why the editors selected it for @eLife.
The goal is to shift people's attention away from the journal title as a measure of a paper's value - which we all know doesn't work - and towards the specific contributions of a given work of science.
If we do this right, we hope it will become a standard for all journals. And while it won't immediately get rid of journal titles, we hope it will begin to undermine them.
Of course this only goes part way, as it will only apply to papers we accept. In the long run we don't want to accept or reject papers -- rather we just want to assess them all by simply saying what the reviewers and editors think of the work without any kind of seal of approval.
Which leads to ...
One of the things that I find most frustrating about pretty much all journals (save those like @PLOSONE) is "triage" or "desk rejects" or whatever you want to call it -- the process by which editors decide whether a paper should go out for review.
If you accept the current journal system, there is a certain logic to this -- if a paper has little chance of being published in a journal even after peer review, it's a waste of everyone's time and effort to review it.
The problem is, of course, that it's really hard to make a judgment about the audience, impact, value - whatever criteria you care about - of a paper without reading and thinking about it in detail - i.e. peer reviewing it.
So the process ends up being incredibly subjecting and immensely frustrating to authors, who feel like their work wasn't given a fair shake. And this subjective pre-judging is really impactful, serving as it does as a gateway to journals that can make or break your career.
A much saner system is, as I said above, to simply review everything and instead of deciding which pigeon hole a paper belongs in, just publish the review along with the paper.
And Initiative 2 is exactly this -- once we get the details worked out, @eLife will begin offering a service by which we will review papers posted on @biorxivpreprint without triage (there will be limited capacity, but it will be 1st come 1st served)
We will then review the paper just like we do papers submitted in the traditional system -- but instead of sharing the reviews only with the authors, they will be posted back to @biorxivpreprint, and will be written to be useful to the public.
Some of these papers will be, perhaps following revisions, published in @eLife, so people can still participate in the traditional publishing system even as they're also stepping into the future.
I realize these may sound like small steps. But they're not. If all journals started to do this we would be significantly further along towards the kind of "publish - review - curate" model I discussed at the beginning.
And the reason I think they will work is that, while they're radical in conception, they're incredibly practical in construction.
There are a bazillion details to work out - but we're working on them, and our plan is to roll these out this year.
And, as always, I'd love to hear from anyone and everyone about these ideas, or other things you think @eLife should be doing to make this vision real.
