We have released the first version of a template README for social sciences. social-science-data-editors.github.io/template_READM…
Joint work with @RevEconStud (@korenmiklos), @EJ_RES (Joan Llull) and @CanEconomics Journal (Marie Connolly). We encourage use at @AEAjournals immediately, and will start to verify usage in May 2021.
We also got input from editors at JOLE, JPE, Management Science (@bgreiner_tweets), and JASA (@ChrisPaciorek) (see JASA's version at jasa-acs.github.io/repro-guide/pa…). We hope to get additional input from others.
Want to contribute to make it better? github.com/social-science…
Want to add your journal to the list of endorsers? Add yourself to github.com/social-science… and send us a pull request.
What is the README for? README in the social sciences serve multiple purposes: guiding a reader through the available material, a route to replicating the results in the research paper, including the description of the origins of data and/or description of programs.
We provide templates/examples for Data and Code Availability Statement: where do the data come from #datacite, access restrictions, how the data can be used. Prove that you are allowed to distribute, help others get access, not just URL! (we provide examples!)
Of course, it contains a (detailed) set of instructions to run the code. But also requirements: do you need a laptop from 2010, or a cloud cluster with 102 nodes? Runs in 5 minutes, days, weeks, months? What packages/dependencies? (we provide examples!)
We note that a good replication package uses a minimal number of automated scripts, with no manual interventions required unless absolutely unavoidable.
The template README has been battle-tested in our attempts (across all of our journals AEA/Restud/EJ/CJE) to assess (computationally, or just reading) the READMEs of more than 500 articles. If we suggest you provide the information, it's because it is useful!
Argh! Forgot @p_morrow_UofT ! Sorry!
One recent example: In the README, we ask authors to provide details on software, including versions. Random seed should also be set (missing from the README github.com/social-science…, but required by most journals) 
In one recent paper with @JuliaLanguage , authors did not specify version, we ran with 1.5.1 (noted on our report), and found discrepancies - despite random seed being set. Authors kindly pointed out that there's a change in RNG in Julia - leading to (slightly) different nums
All the details are important - you just never know when!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
