This paper is misleading as they solicited grant applications from investigators and their entire set consisted of funded grants. So they did not have any poor quality grants (or even any that did not get funded) in their pool.
https://twitter.com/BreznauNate/status/1409934931720085508
While the NIH process is not perfect, I am impressed by how carefully reviewers read proposals and know that my colleagues put a huge amount of effort into grant reviews.
Usually when I'm on a panel, I find my scores are not too different from the other reviewers, but when we disagree, we get the chance to air our disagreements in front of the rest of the room and everyone else can decide for themselves what they think.
Early on in my career, I was puzzled by why tenured faculty work so hard. What is the incentive? And then I figured it out. Getting grants. And to get grants, scientists have to perform at the very highest level. It is super competitive.
But I could be wrong, as I have no idea how to account for work-addicted tenured humanities professors. What is going on there? Do we all just get hooked to the positive reinforcement of getting our work out there?
Basically, the pool of grants being evaluated were among those previously evaluated as in the top 18% (a typical rate of grants funded at the NIH). This would of course reduce the variance and the ability to detect correlations among reviewers.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
