Thread by @FabioGhironi

A thread on second- and third-order comments as research killers, related to a recent @bopinion post by @tylercowen. 1/n

I concluded long ago that the economics profession has been drowning in second- and third-order comments by journal referees. 2/n

Far too often, reports contain a plethora of requests of ancillary exercises and assorted minutiae, and editors—when they do not reject the paper—are so prone to just requesting that “all comments be addressed” (in the paper and/or in pages-long response documents) 3/n

that we all end up spending *huge* amounts of time doing work that often implies little (if any) change to the actual core idea and results of the paper but prevents us from using that time for something more important—like working on a new project, on a different core idea. 4/n

I remember when I was in grad school, and I read amazing papers that were transparent, short, and to the point. Now papers are like books because we are all trying to preempt referee requests by doing all possible extensions in advance. 5/n

The majority of those extensions end up in absurdly long online appendixes that will be read by a minuscule number of people (if any). Reading a paper has become too often a painful, boring slog. 6/n

The publication process in economics has lost any value as “timely dissemination of peer-reviewed research” because, by the time they are published, papers are old, years old, in some cases even more than a decade old. 7/n

The main role of journal publication in economics at this point is as a stamp of approval for tenure and promotion cases, or to generate outside offers, because departments do not actually read the candidate’s work. Some stamps are worth more than others and that’s it. 8/n

Heck, even assuming that people are not outright lazy, we are all so tired from reading 50 pages every time, that we are more than happy to just say “Hey, he/she has a top 5, the letter writers are in favor. Great. Tenure.” And much too often no top 5 means no tenure. 9/n

Meanwhile, we all continue to drown in extensions, robustness checks, rewriting this or that page, or rearranging this or that part of the paper, moving that section we really liked to that online appendix that no one will read, because one referee did not like it 10/n

(maybe after that section had been the core of 25 presentations we gave in seminars and conferences where plenty of audience members complimented us precisely on it). 11/n

Leaving aside the consequences of our screwed-up publication process for the role of journals in research dissemination and tenure/promotions, maybe the cost of how the system works is more limited for those who have the luxury of very light teaching and advising loads, 12/n

because the amount of research time these fortunate scholars have is large. But for many of us who do not have the privilege of very light loads, the way the system works is literally a killer of ideas. 13/n

We are all chasing interminable appendixes and responses to referees, and we end up having no time to work on ideas that may be an order of magnitude more important than any of those second- or third-order adjustments to our existing papers. 14/n

How many ideas do not get studied because of this? 15/n

How many extensions that could be standalone publications by others (for instance, students learning the ropes of research on the way to their own first-order idea) instead end up buried in online appendixes that no one will read? 16/n

I am as guilty as anyone to have contributed to the current equilibrium. As a referee, especially when young, one feels compelled to give all the comments one can think of. We are genuinely curious about those extensions we’d like to see. Why shouldn’t those authors be too? 17/n

I have come to this conclusion: If you don’t have first-order comments, don’t feel that you must write a list of second- and third-order ones anyway. It’s ok to say “This is a good paper as it is” (and explain why to the editor). 18/n

If a paper is good, it’s good because of its core message, not because of the second-order stuff. It’s ok to publish it even if not every extension, robustness check, alternative data, and whatnot have been explored in those interminable appendixes. 19/n

I am very grateful to editors who are making an effort to push referees in the direction of focusing on a few first-order comments, separate from “suggestions” that the author(s) should be free to ignore. It is the authors’ paper! We need more editors to adopt this strategy. 20/n

Let scholars move on to their next first-order idea. Let someone else pursue the extensions, alternative data, alternative technique, and so on, if they want to. 21/21

I just wanted to thank everyone who replied to this thread, liked it, and/or retweeted it. It was striking to see the support it received. >

Editors and those who appoint them hold the keys. Hopefully, we will eventually move to a better equilibrium.