Thread by @mcxfrank

What is "the open science movement"? It's a set of beliefs, research practices, results, and policies that are organized around the central roles of transparency and verifiability in scientific practice. An introductory thread. /1

The core of this movement is the idea of "nullius in verba" - take no one's word for it. The distinguishing feature of science on this account is the ability to verify claims. Science is independent of the scientist and subject to skeptical inquiry. /2

en.wikipedia.org/wiki/Mertonian….

These ideas about the importance of verification are supported by a rich and growing research literature suggesting that not all published science is verifiable. Some papers have typos, some numbers can't be reproduced, some experiments can't be replicated independently. /3

This research is translated into practice through tool-building and policy. Tools like github.com and osf.io facilitate the sharing of materials, code, and data, which allow analytic reproduction and independent replication to verify findings. /4

osf.io and aspredicted.org also allow preregistration, providing verifiability of analytic hypotheses. Prereg is not about discouraging exploration! It's about being transparent about *what you knew* before data collection. /5

babieslearninglanguage.blogspot.com/2016/07/prereg…

The file-drawer problem (unpublished negative findings) is a major challenge to transparency goals. "Registered reports" provide preregistration plus review & commentary; they increase transparency by ensuring publication regardless of result. /6

cos.io/rr/

Tool building and science translates to policy through organizations like @improvingpsych and journal guidelines & initiatives (e.g., badges). Funder initiatives for openness also are policy targets. /7

cos.io/our-services/t…
grants.nih.gov/policy/sharing…
psychologicalscience.org/publications/b…

But making policy based on scientific evidence is always tricky, and polices may have unintended effects, or reveal further weaknesses. Case study: in one of our studies, open data policies revealed further weaknesses in analytic reproducibility. /8

rsos.royalsocietypublishing.org/content/5/8/18…

Open access is an important policy frontier: without access to the literature, results are only verifiable (or readable!) by those lucky enough to have subscriptions. Preprints and "green" OA initiatives address this issue. /9

academia.stackexchange.com/questions/5199…
psyarxiv.org

Transparency, verifiability (and the skepticism that comes with them) can be uncomfortable. Why am *I* being verified and not someone else? Why don't you trust me? I'm an expert! To me, these feelings are very understandable (I have them myself sometimes). /10

Some ways I gently push back: 1) appeals to the shared project, rather than the individual (we're in this together). 2) appeals to the values (nullius in verba again!). 3) argument from the counterfactual: would it be better not to know we're wrong? /11

sometimesimwrong.typepad.com/wrong/2016/02/…

There is a lot of change happening, and change can be disorienting. But I'm very optimistic about psychology and science more broadly, and very glad that I'm working at this time and not another. /end