,
21 tweets,
9 min read
Read on Twitter
It's official, the Review of Economic Studies does not accept comment papers (unless they point out actual factual errors), according to my correspondence with an editor. In important ways, it's not really an academic journal. @RevEconStud #EconTwitter #EconReplicationCrisis
This isn't hard. If the RESTud doesn't accept comment papers, then people who do publish there now have an incentive to sneak papers past the referees which are obviously flawed. And people don't have incentive to replicate papers, since they cannot be published. #EconTwitter
It's no mystery why the editorial board refuses to accept comment papers. The benefit of being an editor is not the pride one takes in leaving one's intellectual stamp on the field, but rather the benefit one gets from giving favors to powerful people in the field. #Econtwitter
Another benefit of being an editor is that you can publish friends papers, papers in your network, and papers which you like ideologically. Allowing comment papers would invite criticism of your network and those you have traded favors with. #EconTwitter
Another argument for not accepting comment papers is that they are poorly cited. However, given hysteresis, it's unlikely that RESTud will ever drop out of being a "Top 5" journal, even though it should. It frequently publishes bad research. #EconTwitter
It's also interesting that RESTUd is too embarrassed, apparently, to post this policy on its website. Rarely a good thing when an "academic" journal resorts to secrecy. restud.com/submissions/ @TheReplicateNet #EconTwitter
What can be a legitimate intellectual justification be for not allowing replication? To me this is a scandal. It speaks to the heart of whether academic economics is a science or just a bunch of rent-seekers collecting large tuition fees and not adding any value. #EconTwitter
Neil deGrasse Tyson on the essence of science "You have a hypothesis, you test it. I get a result. A rival of mine double checks it, because they think I might be wrong." RESTUD: Sorry, not interested in double-checking anything.
Nowhere is the chasm between the ideals of most economists, and the reality as stark as when one thinks about the #ReplicationCrisis in academic economics. It's the difference between the Japanese concept of the "tatemae" (how things seem on the surface) vs. "honne" (reality).
On the surface, @RevEconStud is a very serious scientific journal publishing meticulously peer-reviewed work. In reality, while it publishes good work, it also publishes loads of crap from people who have a good network & a good feel for a popular story.
Yet, people believe the appearance of seriousness. No replication or not, a single publication there will be taken very seriously come tenure time, and by people who should probably know better.
I can see why the current editorial board does not want to change the status quo -- they will lose out under a change in policy. Yet, why should the rest of us go along with this? Why cite it as a reputable academic journal when it isn't? restud.com/editorial-boar…
In any case, perhaps this is all an over-reaction on my part. To me, this is shameful. Shameless from the editors, and shameful for the profession to allow it. #EconTwitter
Until the #ReplicationCrisis actually happens, I think Economists should stop trying to pretend that they have some moral high ground when compared to higher wage professions like consulting or investment banking. #EconTwitter
Update: I just spoke with a member of the editorial board of @RevEconStud, and they said they would raise this issue at the next board meeting. I am encouraged that they will do the right thing.
However, I also believe that they should also use any debate to implement a forward-thinking policy. The @RevEconStud could use its market power to actually encourage comment papers. Perhaps even commission them on seminal papers.
I suspect what @RevEconStud will do is take the QJE policy. They formally allow comment papers, but seem to publish them only rarely (I just checked the last 6 issues, and saw none). Here was my own experience at the QJE: statmodeling.stat.columbia.edu/2015/12/19/a-r… #EconTwitter
One reason to favor Restud's current policy is that it makes it clear it cares little about science. By its own admission it doesn't accept any criticism of the papers it publishes! Once it switches to a QJE-type policy, it will merely appear more serious while little changes.
Economists rarely praise the AER. But, when it comes to replication, the AER has been something of a godsend. It was an early adopter on the data policy, and frequently publishes comments. Way above the likes of the QJE or RESTUD. #EconTwitter
The other journal that deserves high praise is the Journal of Applied Econometrics. It's the clear leader in replication. replicationnetwork.com/2018/10/31/ree… #EconTwitter 
But, what stands out to me is this: only 26 journals have published a replication since 1967! It's not clear to me all comments count as "replication" here, but no economist should be proud to read that. We can do better. #EconTwitter
