The #metascience2019 meeting got me thinking about reproducibility in modeling. From now on, in collaborative modeling projects, I'm going to have two individuals reproduce the same results separately. A thread.
Shirley Wang had trouble reproducing results from big data sets trying to reimplement the code others developed.
In my own work, I've caught little errors last minute, and once had an error in a published paper. I've caught many errors in student code.
It's easy to make a mistake in even relatively simple simulations, much less more complicated ones. In addition, there are often small misunderstandings in the team about how details are being implemented in an algorithm.
To sum, benefits 1) fewer errors, more confidence in results, 2) more clarity within the working group about exactly what is being modeled and so 3) hopefully clearer communication about exactly what the model is.
And so I've spent all of today reimplementing the simulation in a paper my students did this last year.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
