I see a lot of complaints in my Twitter stream about academic conferences rejecting papers. I think many somewhat miss the point - a lot of unfair rejections are a direct result of the current incentive structures that generate too many papers and focus them on too few confs.
Conferences are more reputable the more selective their are. This means that if a conference received too many submissions to sanely handle last year, they will receive even more now.
The modern PhD process tells a bunch of bright young researchers “the publication of N papers at top-tier conferences is a requirement for you to get a PhD” - without telling them what useful problems are, and nowadays even discouraging people from doing non-superficial ...
... literature review. It also puts time pressure on them. The result is that (A) the vast majority of papers contain the absolutely least publishable unit (the most marginal idea that may get accepted), (B) often tackling wrong problems that are not chosen by importance but...
... by convenience, (C) the papers have to use hyped-up language to make sure they look good and have to overstate their utility, (D) a wrong-headed focus on “novelty” vs “does this actually work and is true”, (E) a culture where a paper is successful if it gets past a PC vs ...
... where it is successful if it is doing good stuff.
This creates multiple vicious cycles. (1) Non-academic reviewers (such as me) get frustrated and burnt out by a flood of marginal papers. I love good research, but am about to drop out of academic reviewing entirely because he reviewing is getting less and less rewarding.
This exacerbates the “no feedback about what problems are important” gap. (2) As reviews get less rewarding, and as paper submitters increasingly ignore longer feedback and simply resubmit the same paper to the next conference, reviewers pull back on time spent and review detail.
(3) Hyped-up language makes everybody discount claims across the board, which then punishes those with a more honest and sober writing style. The hype-language is getting worse over time, as is the novelty overfocus - I recommend comparing papers form ...
... 15-20 years ago to today’s just on style.
The reforms to the system are needed are not happening because the system selects for those that thrive under these conditions, and then places them in powerful positions where they are much less exposed to the downsides.
The reforms to the system are needed are not happening because the system selects for those that thrive under these conditions, and then places them in powerful positions where they are much less exposed to the downsides.
I love good science, and a nice paper that explains a new idea to me. I read other ppls PhD theses for *fun* in the evenings. I am about as academia-friendly as any practitioner, and am about as theory-affine as any practitioner. I value long-term research very highly.
The fact that people of my ilk are getting frustrated away from dealing with the broken incentives of the academic system should be worrisome - but only if one thinks hat the output should be science, not paper counts.
