I suspect that peer review *actually causes* rather than mitigates many of the “troubling trends” recently identified by @zacharylipton and Jacob Steinhardt: arxiv.org/abs/1807.03341
I frequently serve as an area chair and I manage a small research group, so overall I see a lot of reviews of both my group’s work and others’ work
It’s very common for reviewers to read empirical papers and complain that there is no “theory”. But they don’t ask for theory to address any specific question. I think they are just looking for an easy reason to reject—-they skim and don’t see scary equations.
This is easily addressed by adding useless mathiness. Reviewers generally don’t call it out for being useless. It passes the “I skimmed and saw a scary equation or pretentious theorem name” test
Similarly, reviewers often read a submission about a new method hat performs well and say to reject it because there is no explanation of why it performs well
If you do add an explanation, no matter how implausible or unsupported by evidence, that’s usually enough to placate reviewers
Reviewers often see papers that use empirical observations to understand how a system works, and respond with complaints that there is no new algorithm. This is easy to address by throwing a practically irrelevant new method into the paper.
Reviewers seem to hate “science” papers, but it’s possible to sneak science in the door if add some token amount of new method engineering
(This last one is a bit less consistent than the others. I’ve seen a few science papers get high review scores without having to sell out... but they often get rejected from a few conferences before being lucky enough to get reviewers who get it)
Some of the other troubling trends would probably happen without peer review, but I see reviewers basically asking to add mathiness, spurious explanations, and spurious novelty all the time
