Thread by @sTeamTraen

Thread:

It seems to me that one of the best things that anyone can do to advance the cause of open science on a daily basis is through our peer review activities. /1

Some people have signed up for #PROinitiative. But even if it feels a bit confrontational to demand the data, you can still ask questions of the authors. For example, can they provide a full table of descriptives (means, SDs, correlations)? /2

Have they reported all the conditions that they tested, and all of the dependent and independent variables for which they collected data? Are they prepared to put a statement to that effect in the article? /3

When you see an analysis with a covariate that gives p=.037: Ask for the same analysis without the covariate to be included in the article too, along with some reasoning for the the inclusion of the covariate (especially if the version without gives p=.056 😀). /4

I used to worry a bit about asking for this sort of thing, because (AFAIK) currently most other reviewers don't ask for it. It almost felt unfair to impose these unusual requests on the "unlucky" authors who happened to get me as a reviewer. /5

But now, I think: We have to start somewhere. There probably isn't going to be much change coming from the top any time soon, with one or two exceptions; and if or when it does come, it will be because of the shaking of the tree at the bottom. /6

Regarding anonymity, I would say: Don't worry too much if your impostor syndrome (versus "being a dick"; don't do that) makes you feel unable to sign your reviews. On balance I feel that signing is probably "better" (for some "good"), but I only sign if the journal insists. /7

To sum up: Maybe you can't do a lot on your own. But individual contributions like this can add up; in fact, they're one of the few ways that people without power can change anything, They can also be good for your own morale: "I did something to improve science today". /end