Thread by @SfPRocur

Hi! I'm back.

I’ll get back to Science Reading shortly. Apologies for leaving you hanging. There will be much more later today.

For now, as promised, we’ll start a separate thread on our "QRP Reviewer Quidelines" project. Joining us for the description & discussion is @RemiGau

The lack of training when it comes to reading scientific articles goes hand in hand with a lack of training on how to review articles: it is often assumed that one learns by doing it.

Given the huge pressure in academia to publish evermore new, positive results & in the most prestigious journals, often at the expense of good quality science, this lack of training can have dire consequences when researchers engage in questionable research practices (QRPs).

QRPs live in the grey zone between what is considered good scientific practices and scientific malpractice (fabrication, falsification, plagiarism,...).

They typically involve practices such as p-hacking and harking

See this paper for an overview of QRPs:

Questionable Research Practices Revisited by Klaus Fiedler and Norbert Schwarz
mfr.osf.io/render?url=htt…

QRPs happen when scientists rely on what are called researcher degree of freedoms (the many ways experiments can be designed, run or analysed) to find new, surprising, positive results.

Addressing the problem of QRPs usually involves

a) prevention by raising awareness amongst researchers

b) post-publication correction either by closely examining “suspect” published studies or by attempting to replicate influential studies.

This leaves the middle option of trying to deal with possible QRPs in studies that are currently in the pipeline.

The idea behind the “Reviewer Guidelines for QRPs” is to correct these errors BEFORE the science becomes part of the literature.

The question is how best to do this given that:

1. This is a massive topic. There are many QRPs, and much literature about them.

2. A problem with better science practice movement is the top-down effect where many people are resentful of norms seemingly imposed by a select few.

3. It would be a massive amount of work for any single person to take on.

What we propose then is crowd-sourced set of guidelines describing what reviewers should do when encountering possible QRPs.