Profile picture
Michael C. Frank @mcxfrank
, 9 tweets, 5 min read Read on Twitter
Everyone makes mistakes during data analysis. Literally everyone. The question is not what errors you make, it's what systems you put into place to prevent them from happening. Here are mine. [a thread because I'm sad to miss #SIPS2018]

A big wakeup call for me was an errror I made in this paper:…. Figure 1 is just obviously wrong in a way that I or my co-authors or the reviewers should have spotted. Yet we all missed it completely. Here's the erratum.…
Since then, we've audited dozens of papers (I like this term much more than "data thugged" @jamesheathers). E.g. in @Tom_Hardwicke's new manuscript:…. Summary: the error rate is very high. Most errors don't undermine papers, but most papers have errors.
So what do we do? 1) Literate programming. If you have to write code that others can read, you catch typos and errors much more quickly. And if you're not scripting your data analysis using code, it's time to start.…
2) Standardize your workflow. If you do things consistently, you will be less likely to make new, ad-hoc errors that you don't recognize. For me this has meant learning - an amazing ecosystem for R data analysis.

My tutorial:….
3) Version control. is a gateway. git and take time to learn but are far better tools for collaborating. If you track what you did you will not lose files/erase work/accidentally modify data, blocking a whole set of possible errors!
4) Code review. I wish we did this more. But co-working (…) and pair-programming can help catch errors and promote clean, standardized analysis. And you can always ask a friend to "co-pilot" after you are done.
5) Openness by default. If you curate and analyze data as though someone who thinks you are wrong is watching you and criticizing, you will be much more careful. This is a very good thing. More thoughts in our transparency guide:
In sum: of course we should be afraid of errors! But don't *just* be afraid. Accept that you *have* made errors - and will make more. We all do. Act to put systems in place that help you catch and correct errors before they enter the literature. [end]
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Michael C. Frank
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!