Back from a great #apls2018. Lots of cool research, and I learned a lot. However, I did notice that almost no one referenced #openscience. A thread about why it’s important 1/N
By open science I mean: open materials, open data, and pre-reg (any and all combinations). I went to a ton of talks, and only saw two that referenced it. s/o to exceptions from @RBMichaelPsyc and Aria Amrom who pre-registered via @OSFramework. 2/N
I'm a bit biased b/c all of my work is open/pre-reg’d. All projects in the symposium I organized are open/pre-reg’d. We posted OSF links in our slides. I’m not bringing this up to brag or get credit. I get #openscience is hard, n hope to start discussion, not chastise. 3/N
(Shameless plug: Why Memory Matters: Problems (and Solutions) in Criminal Investigations. osf.io/8dfn5 slides and project links included. Sorry @dstephenlindsay for not making it public earlier! Feedback appreciated! @RabbitSnore @f_alceste @cardenas_s_a 4/N)
To the extent that good science is open science (extensively, I think), lack of open science at the conf bugs me – suggests the work presented isn’t produced with best practices. Now, maybe people are doing it but not posting it. But even not mentioning it is still a problem. 5/N
For open science to be really effective, it needs to be the new norm. This means more than just journals giving badges (tho props to LHB for getting engaged) - needs to be at conf too. It needs to be expected, tacitly rewarded, and perhaps overtly rewarded as well. 6/N
Open science takes work. It takes time, effort, planning, and a bit of bravery. It carries a cost that is necessary for good science. But is there a reward? Maybe people respected our symposium more, or maybe a job search member noticed my work and I’ll benefit down the road 7/N
But maybe not. If I DIDNT open science my work, I’d see no reason to change: I can publish faster, perhaps worse but more exciting=publishable work, if I save time and aim for journals that are still accepting non-open work (which I assume is most) 8/N
If the societies/confs don’t embrace it, emphasis on journal pubs is too down stream. In job apps, is it likely a search committee is going to 1) care about open science, 2) notice which pubs are in open science-emphasized journals and 3) value those pubs over non-open pubs? 9/N
quotes an SPSP panel on student perspective of open science - this thread discusses some of the high costs that current young scientists can suffer from doing things the right way.
10/N
10/N
Societies, like @APLS41, should be valuing open science at the ground level of empirical proliferation: conf presentations. Senior members should be encouraging it in their labs before confs, and valuing it in work at the confs (thnx to @DerynStrange for doing exactly this) 11/N
Large scale, this makes #openscience the norm. So even if there aren’t direct benefits (e.g., greater chance of talk/publication acceptance, tho see below) there are professional boons (collaboration, attention drawn to work, greater visibility of your best practices). 12/N
That’s why even if psychologists are doing open science but not including it in presentations, it’s a problem that it isn’t being publicly recognized. Should be self-promoted (brag on yoself!), promoted by members, and by leadership 13/N
If I’m a grad student, I learned nothing about an essential component to conducting research. Yet we are supposed to learn about conducting research. Many talks advertise new paradigms at confs, so the message is: what paradigm is important, but process is less important. 14/N
If I’m an early academic/ ECR who doesn’t do open science, I see no reason to change what I’m doing. Either consciously ("why do all this extra work?") or inadvertently by never seeing #openscience modeled or mentioned ("what's open science, anyways?") 15/N
I’ll admit I’ve never organized a conf before, but a few solutions strike me as easy and painless: 1) talk/poster applications encouraging open science work, with a box for links (eg, OSF links) not part of the word limit. Publish those in conf programs with abstracts. 16/N
That will promote visibility of the work AND the practice. Function AND fashion. Hell, the Haiku Crew (@JeffKukucka @jenperillo @ADPerillo) can tweet their haikus and include the 10-character link right after their haiku (they def have the character space) 17/N
2) organize workshops/webinars/guides/accept symposiums on #openscience practices. Pre-conf and at-conf. Could be tailored for specific fields, such as exp materials for case materials, confessions, eyew and lineup stimuli. I bet would increase collaboration and citations 18/N
3) consider open science practice recognition for talks and posters. Could be student-specific, @APLSsc. 4) encourage (require?) open science for confs and submissions. Even just posting slides: shallow use with the process would, I think, really help. Empirical question tho 19/N
To clarify, I’m not here to criticize. #Openscience can be hard, overwhelming, and a bit scary - esp for students. There is variation in what can be made available. It takes time to learn (you could be writing!!) so I’m not saying strict, exclusionary guidelines. 20/N
But in my experience, open science wasn’t even a discussion point at the conf - and it should be. I hope that next year, I can have spontaneous conversations about open science with people who aren’t my normal collaborators - comparing stories, practices, and projects 21/N
I think it would be good for both @APLS41 , and the individual members, if presented research was expected to be open and transparent, and was produced with best-practice. END
Note: Curses! That osf link is to the first project. Here's to the overall symposium: osf.io/hm5vn/
