Thread by @jamesmfinley

Educating the next generation about incentives and rigor in science, a thread

One of my teaching responsibilities at @USCBKNPT is to direct a course for 1st-year graduate students on experimental methods for analyzing human movement. (1/11)

Over the past few years, like many others who teach research methods, I’ve been integrating more content about reproducibility into the course. (2/11)

This began with a discussion of Estimating the Reproducibility of Psychological Science
science.org/doi/10.1126/sc…
and this yr, I added The Natural Selection of Bad Science to discuss how the incentive structure in science may select for poor methods
royalsocietypublishing.org/doi/10.1098/rs… (3/11)

One of the highlights from our discussion of the @psmaldino and @rlmcelreath paper was when the class was asked to generate ideas for how they might change the incentive structure to encourage the selection of more rigorous science. (4/11)

The prompt involved having the class consider changes that they would make at the level of graduate education, at the level of faculty hiring, and in setting priorities for research funding. (5/11)

I was really encouraged by the students’ willingness to critique the current system and brainstorm solutions for a better future. Solutions included 1) having more discussions about rigor, reproducibility, and incentive structures as part of graduate-level education, ... (6/11)

...2) evaluating potential faculty hires in part based on how they demonstrate evidence of rigor in their work (with less focus on the number and venue of their publications), and 3) prioritizing the funding of research that explicitly contains a replication component. (7/11)

There were other insightful comments that I was really excited to share last week when the details were fresh on my mind, but hey, who has time to post more than one thread in a week? (8/11)

The take-home point is that students are clearly ready to discuss these issues and imagine a better future, and I strongly believe that these discussions should be a requirement in all graduate-level (and even undergraduate) education about research methods. (9/11)

The bigger point that has stuck with me is that many of us ARE the system and have the power to implement change at many levels (our labs, grant review panels, hiring, mentoring of junior faculty, evaluation, etc.). (10/11)

I suspect that I'm largely preaching to the choir here, given the nature of people who tend to engage on science Twitter, but hopefully, someone sees this and makes a change for the better in their own sphere of influence. <steps off of soapbox> (11/11)