Thread by @pereztruglia

🚨 Tips for writing a POSITIVE referee report.

As an author and an editor, I’ve noticed some reviewers are better than others at writing a positive referee report. So, I decided to share 3 simple tips in the thread below. 👇

#EconTwitter

Tip #1: Act fast!

If you love the paper, submit your report ASAP. Editors see their decisions as an optimal stopping problem. They invite more referees than they expect to need, and skim through the reports as they come in. So, the order in which reports are submitted matters.

Let's say an Editor invited 3 referees for a paper. Those referees would (eventually) recommend: 1 Reject and 2 R&Rs. If the editor waited for all 3 referees, the decision would be Revise. Let's see what is likely to happen when the reports are submitted in different orders.

If the 2 R&Rs are submitted first, the editor will most likely stop and invite a revision. If Reject comes first, the editor may stop and reject. Even if the first two referees are mixed (1 R&R and 1 Reject), the editor is likely to stop and reject. So the order matters a lot!

Tip #2: Show awareness of the weaknesses.

Positive reviewers sometimes send a short report just saying that the paper is above the bar. But let’s say the other reviewer recommends rejection based on specific concerns (e.g., the identification strategy is not persuasive enough).

This leaves the editor wondering: did the positive referee read the paper carefully? Did this reviewer thought of the omitted variable bias mentioned by the negative reviewer? What about that p-value from column (3)? The editor may take the safe route and reject the paper.

Show the editor that you are aware of the manuscript’s limitations. E.g.: “The paper does not use an experiment, so the causal identification is far from ideal. However, this is the very first time a paper was able to do X. That’s an impressive feat that others will follow.”

Sometimes you even know what other people hate about the paper, but you disagree. Perhaps you saw a presentation of the paper, or you refereed the paper before for another journal. Bring it up! E.g., “I think other reviewers may say X, but I disagree with them because of Y.”

Tip #3: Show your love!

If you love the paper, say it. This is the language I encounter most often: “The research is carefully executed and the paper is well-written.” This far from enough to get the editor excited about the paper, specially if the editor is from another field.

For instance, sometimes I read a paper and the first thought that comes to mind is: “This is so cool, I wish I came up with this idea!” If you have the same thought, just tell the editor. This will send a strong message, specially if the editor thinks highly of your own research.

More examples: “I tried X in the past, and I could never pull it off. I’m glad the authors did!” “I can’t believe there are hundreds of papers on this topic and nobody did X before. I’m glad the authors did!” “Before reading this paper, I thought X. Now I changed my mind.”

If you have other tips to be a positive referee, please respond to the thread. And remember that we are just humans doing science. So, regardless of whether your recommendation is positive or negative, be kind always!