On the back of the Voynich Manuscript story, especially for those outside of academia, I wanted to shed some light on the issues that have allowed this story, and others like it, to happen. In the beginning, you submit an article to a journal relevant to your topic... 1/20
At that stage, the editor will often give it a quick skim and decide whether it's even fit for peer-review. If not, you get the ultimate humiliation of a desk-reject. Urban legend has it that the fastest desk rejection in history took fourteen minutes and came from @Nature. 2/20
Even if that’s true, I’m sure @ScienceMagazine won’t be outdone for long.

If the article gets over the desk, it goes out to peer-review, or in other words, it’s sent on to other academics, who are usually specialists in the topic in question, for them to review and assess. 3/20
Articles can be peer-reviewed by as few as one person, and as many as six or seven, and reviews can go through multiple rounds. Typically peer-reviewers suggest rejection (F, wonk wonk), major changes (C-), minor changes (B+), or, rarely, accept as is (A+, overachiever). 4/20
Normally reviews are double-blind – the author doesn’t know who is reviewing their work, and reviewers shouldn’t know who the author is. In practice, the reviewer might guess, but morally and ethically, authors/reviewers shouldn’t try to identify each other. 5/24
The plus side to double-blind peer-review is that it theoretically stops people from gaming the system by sinking the work of their enemies and promoting the work of friends and "big names in the field". The downside is something known as the Reviewer 2 phenomenon. 6/24
Sometimes, reviewers are utter bastards and they write heartbreakingly awful feedback that sends you face-first into a gallon-bucket of cheap icecream for the rest of the week. And for some reason, it seems to usually be Reviewer 2. I had one such just like this. 7/20
It included a dig which revealed that the reviewer had deliberately found out who I was so that they could make their comment quite personal. Editors should screen reviews beforehand so we pointed out this comment to ours, and they apologised and blacklisted the reviewer. 8/20
Anyway, the system isn’t perfect, and there are other problems besides. Sometimes the article is so niche that there are no perfect peer-reviewers to cover every aspect, so each reviewer can only review certain sections, whilst making informed judgments about the rest. 9/20
Peer-reviewers also don’t get paid. It’s totally voluntary and takes hours. Unsurprisingly, some academics refuse to peer-review full-stop. Some accept reviews but do a poor job. Some review work outside their area, but their good intentions don't make rigorous reviews. 10/20
Let’s turn to this Guardian quote: “Asked for his response to those who were unconvinced by his interpretation, Cheshire was bullish. “The journal paper has been blind peer-reviewed and verified by other scholars – that is standard confirmation in the scientific arena.”” 11/20
(That was from the Guardian by the way: theguardian.com/science/2019/m…)

Hopefully you can already see some of the problems here. Just because it’s been peer-reviewed, that doesn’t mean it’s right. The editor and the reviewers are human, fallible, and capable of making mistakes. 12/20
Unfortunately, if even editors and peer-reviewers can get it wrong then what hope does the media and layperson have? Pretty much none, unless they just happen to have a PhD in that exact topic. And, there is also a general misunderstanding of what it means to be published. 13/20
The idea is that you work on a theory and then, literally the very first and lowest step towards getting your work out into the academic community, you publish it in a peer-reviewed journal. Once there, it then faces FULL scrutiny from *all* of your academic peers. 14/20
And that academic community is well within its right to say, “Vaccines and autism wait what now?” Or “How are you going to write a grammar of a language whose writing system you don't understand? How can this possibly have been peer-reviewed?” a la @BenJCartlidge. 15/20
In fact @BenJCartlidge peer-reviewed this article live today. Here's the thread: You're wecome.

But anyway, this should also explain why sites like @RetractionWatch exist. Because sometimes rogue research gets loose, runs amok, and needs shooting. 16/20
So what may have happened is the article somehow made it across the Romance Studies desk, got through peer-review (we will likely never know who reviewed it or what they said) and once in print, that was enough for non-specialists to believe that it was valid. 17/20
From there the @BristolUni press team ran with it, since they are no more mediaevalists/codicologists/linguists/cryptographers than the average person, and they likely put out a ton of very excited and well-meaning press releases across all their platforms and networks. 18/20
Then, by the time they got any sense that the story and claims were highly problematic, by that point, it was already too late. Only one big outlet has to break the story. The rest will cannibalize it from them, and by that point, the damage is already done. 19/20
The moral of the story is, take care, oh university press offices, what you promote.

For what hath beene twote, yea but it cannot be untwote.

/fin

20/20
wait that last tweet rhymes wtf
Full thread on one page: threadreaderapp.com/thread/1129011…
In case anyone is dying of curiosity about what Reviewer 2 said, from memory it was roughly, "the author's seniority in her faculty makes me question why she has not had the sense to use [random pet theory]". Reviewer 1 thought the work only needed minor revisions, sooo… 🤷‍♀️
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Dr Claire Hardaker (she/her)
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!