Thread by @pierre_azoulay

OK. I have spent much of the week finally sinking my teeth into this paper. It's really well written and convincing. But the headline result as cast a shadow over some nuances that the authors talk about but have been ignored in the commentary.

https://twitter.com/maxdkozlov/status/1610672547267215362

The authors do say and show that the result is much less pronounced for the life sciences. That's the sandbox I like to study so, ignoring some unimportant caveats, I have replicated their results for the PubMed universe. /2

I have applied the argtanh transformation (because the d index is distributed over the interval ]-1;1[) so that regression results roughly yield estimates in terms of % decline. I have also absorbed number of author effects (from 1 to 30),... /3

...and in some specifications, journal effects. Also I have limited myself to original work (I dropped reviews, letters, editorials, etc.). See below. /4

It's definitely going done, but a 5% decline in the index over 50+ years is maybe a bit less than earth shattering. /5

And yet the results have set the heart of biomed twitter aflutter this past week. If I has the summarize the reaction, it would be "duh! The NIH makes us do it." And I have a lot of sympathy for that view. /6

really, I even have a new paper that documents how risk-taking might be penalized by NIH study sections at the time of R01 grant renewal (more on that later). But here is a little inconvenient fact. /7

I look at the subset of PubMed papers with all US teams, and separate those papers that acknowledge NIH support vs those that do not. /8

And yes it's still going down, but really the effects are small. And if there is an effect, it is *less pronounced* for papers acknowledging NIH funding. /9

None of this implies that the authors are wrong. But the area where disruptiveness has most declined are the social sciences. So we should probably look for explanations that are specific to that domain. /END

And sorry for the typos, I am doing this quickly.