Thread by @betanalpha

Unpopular opinion: not everyone should do their own statistical analyses, and that should be okay! Forcing unprepared practitioners to develop their own analyses sets them up to fail and reinforces an ecosystem of fragile scientific claims. A thread.

As our scientific questions become more challenging the analyses become more demanding of not only our domain expertise but also our statistical and computational expertise. Those not well versed in these will be ill-equipped to build their own analyses. And that’s okay!

In order to sustain robust science on their own practitioners would have to learn the statistics, or statisticians would have to learn the domains. Ideally the two could even work together to contribute their individual expertise to a robust analysis.

I think most people on all sides of contemporary debates about statistical analyses miss this point, which helps everyone talk past and frustrate each other instead of actually improving anything.

For example introductory classes that teach a few statistical and computational basics are great! When done well they help practitioners make their first step towards statistical understanding and, perhaps more importantly, improve their ability to communicate with statisticians.

At the same time they are nowhere near sufficient instruction for developing an entire analysis. In other words they can be counterproductive if they don’t also establish the limitations of the techniques taught and foster overconfidence in what the students are prepared to do.

I think that many complaining about intro classes are focusing on that latter issue, where practitioners leave the class thinking that they’re ready to use sophisticated tools for which they are in reality woefully unprepared.

But if you’re complaining about bad teaching then _provide better pedagogy yourself_! At the very least create material that demonstrates how subtle and dangerous these methodologies can be so that practitioners are aware of their limitations.

At the same time strive to make it clear that they can learn to work with these limitations with further study. Otherwise you’re just being a hypocritical gatekeeper and facilitating irresponsible use of tools.

We need to move past the delusion that everyone should analyze their own data no matter their preparation. We need to celebrate those practitioners who recognize their own limitations and, instead of forcing an analysis with tools they don’t understand, reach out for help.

We need to develop better pedagogy and tools for those practitioners with the desire and _opportunity_ to learn the necessary statistical methodology. We need to train, sustain, incentivize, and reward _applied_ statisticians who invest their time in domains.

Both sides need to celebrate collaboration between domain experts and statistical experts beyond empty promises in grant proposals...

We need to celebrate and incentivize practitioners who careful collect, curate, document, and share data even if they don’t analyze it. The same for practitioners who don’t collect data but rather take up the baton and develop robust analyses in collaboration with those who do.

Everyone should have the _opportunity_ to participate but no one is _entitled_ to the entire analysis without investing the requisite time and effort. Everyone’s contributions are necessary for a good analysis, but few are sufficient.

Cheers to those exceptional people who are fighting the perverse incentives and working towards this goal from both directions. We may not fix the system but at least we’ll get some awesome science out of it. 🍻