Peer review is an imperfect tool, and should not be treated as a Swiss army knife (always applicable). It has its uses where it works and adds value, but there are other uses that are simply not helpful. Primarily, #peerreview can be used to assess empirical studies by finding
potential problems with the data and statistical analysis. Here more perspective can uncover non-obvious errors or limitations that the authors have failed to identify or, also not common, chosen not to present. The questions peer review can ask here are whether the analysis was
done properly, if the results may be reliable, and what they add to our cumulative knowledge. The review focuses on whether the authors have done their due diligence and have done enough to exclude errors in the design and check for errors in carrying out the study. But there are
other types of research, such as social science theorizing that is not inductive but primarily deductive reasoning, where peer review cannot be about finding the objective or empirical meaning (because there may be none, at least not yet). Such scholarship, producing new ideas
for how to think about the world, are more about persuasion, interpretation, and novel lines of thinking. As a result, they are (and should be) attempting to poke holes at what we thought we knew by generating new ideas worthy of pursuing. Here the use of peer review, meaning
known and experienced scholars in the field are asked to assess a paper's 'worth', is a dangerous tool. Because novel ideas challenge the status quo, which is where experienced scholars have a dog in the fight--the very dog being challenged. It requires a significant degree of
sincerity and open-mindedness for these reviewers--unpaid and anonymous, who are the very victims of the novelty, and with little time to analyze the paper in depth--to give the paper a fair reading. Consider an example from personal experience. A paper argues a new boundary
between two fields, which thereby gives more explanatory space for one body of theory at the expense of the other. An editor with a legacy squarely in the latter is assigned, who then invites three reviewers who are all (and quite obviously) active in the same field. All four of
them, editor and reviewers, have a dog in the fight--the same dog, the one being challenged by the paper. The paper is swiftly and, as one would expect, unanimously rejected. Several reasons are provided, most or all of which are from the point of view of and in defense of the
challenged field. There may be valid reasons to defend that field, and for rejecting the ideas in the paper, but it is difficult to see how the arguments can get a fair hearing from a group of scholars who, without exception, would lose if the paper is published and taken
seriously. Consider if this happened to ideas such as geocentrism, evolution, relativity, etc. (No, I'm not comparing myself to Galilei, Darwin, or Einstein.) New ideas are typically very easy to 'shoot down', especially from within the safety of an existing paradigm. It is the
task of the editor to give the new idea a fair hearing, without which we all--the scholarly community as well as society overall, where the ideas may be applied--lose out of potential benefit. In fact, the proper role of the editor should here be to *encourage* the idea, perhaps
even help the author make the argument as persuasive as possible. At a minimum, the idea must be assessed on its own merit and potential, and not evaluated from the perspective of the existing (challenged) paradigm. Our understanding of the social world is based on the novel and
provocative new perspectives that have been published. I'm reminded of e.g. Ronald Coase's editorship of The Journal of Law and Economics, in which he is said to have sought out good novel ideas and encouraged and worked with authors to formulate the best possible argument. This
is rather the opposite of typical (and above exemplified) defensive, turf-protecting peer review. It does not lessen the contributions. Instead, peer review, when not used with utmost care, becomes a gatekeeping mechanism that effectively stands in the way of novelty. The problem
here is how editors can be enticed to act selflessly, and sometimes in direct conflict with their own status and legacy, for the benefit of the greater good. They have the ultimate power, there is no appeals process, and they have little or nothing to gain from taking a chance on
new ideas instead of safely treading in the main stream. They also do not have the time to read every paper carefully. So how do we solve this problem? One possible way is to use open peer review, or publish papers side-by-side with the reviewers' comments as dialogues. Another
is to have scholarly journals take explicit theoretical positions to make it explicit that they publish from a specific perspective (much like newspaper editorials). But it is highly problematic to blindly push theory papers through peer review with the assumption that it is a
proper means for assessing new ideas. Novelty always suffers from not already being fully refined and extensively tested. It will and should challenge the status quo. That is kind of the point.
