Thread by @random_walker

I used to share the popular view that creating and maintaining research software is pointless — no one will use it, you won’t get credit for it, and it will die once the lead author completes their PhD.

Here’s the story of how I changed my mind and what I learnt from it.

Seven years ago my team started a project to uncover creepy online tracking. We quickly realized we’d stumbled into a vast cesspit, and that we were going to be working on it for a while. We figured we’d save time by automating much of our workflow.

In retrospect, that was a good reason to build research software — do it if you think it will be worth your time even if no one except you and your collaborators ever use it. If other people do find it useful, that’s a bonus.

My team members @s_englehardt and Gunes Acar were/are the main developers of the tool. They are far better software engineers than I am. It ended up being stable & usable enough that others started using it without needing our help. This amazed me.

@s_englehardt I learnt that research software is more likely to succeed if (unlike me!) you’ve built software outside the research context. That's one reason why spending a couple of years doing software engineering gives you a leg up compared starting a CS PhD straight out of undergrad.

@s_englehardt It helped that we publicly released our software right away without waiting to collect our academic brownie points (a publication). By the time we wrote up the accompanying paper, there were many studies that used the tool, so we were able to present it as a mature system.

@s_englehardt Happily, we did get academic credit for it. We presented both the design of the software and our main findings in a 2016 paper (webtransparency.cs.princeton.edu/webcensus/inde…), which has been cited a bunch and was selected by @futureofprivacy as one of the 2016 Privacy Papers for Policy Makers.

@s_englehardt @futureofprivacy I’m especially happy about the fact that even after @s_englehardt completed his PhD, the software continues to thrive. His new employer, Mozilla, has found it useful and has taken over its development
freedom-to-tinker.com/2019/04/12/cit…

@s_englehardt @futureofprivacy Meanwhile, over 30 studies have used the software webtransparency.cs.princeton.edu/webcensus/inde….
Its use has expanded beyond studying online privacy: we are wrapping up a year-long investigation into “dark patterns”, led by @aruneshmathur, and I’m excited to share the results soon.

@s_englehardt @futureofprivacy @aruneshmathur In summary, I think building reusable research software is a totally underrated. I don’t mean to downplay the sweat and toil, but the payoffs are worth it, and the positive externalities make you feel good!

@s_englehardt @futureofprivacy @aruneshmathur P.S. Our creepiest findings using the tool are in our "No boundaries" series of blog posts, in which we explored how third-party scripts on websites have been extracting personal information in increasingly intrusive ways:
freedom-to-tinker.com/2017/11/15/no-…