1/ THREAD: You can go to prison based on software: breathalyzer software that says how much you've had to drink. Risk assessment software that says how likely you are to reoffend. Statistical genotyping software that gives confident conclusions where human analysts fear to tread.
2/ Almost all of this code is secret. My colleague @jlkoepke wrote about this last year in the context specifically of DNA mixtures -- medium.com/equal-future/s…
4/ There are many problems here: Courts, defense counsel, and even law enforcement can't check for mistakes. And for most of these tools, the evidence that the tool actually works comes from vendors themselves, not from credible third parties.
5/ Thank goodness, there's growing scholarly and policy literature about this today.
6/ @RebeccaWexler has an excellent paper arguing that such trade secret claims do not, generally, belong in criminal courts: papers.ssrn.com/sol3/papers.cf… @elizabeth_joh has described how the firms behind such tech play an undue role in shaping criminal law papers.ssrn.com/sol3/papers.cf… 
7/ The main argument that justice system vendors make in favor of trade secrecy mirrors the interests of trade secret holders in general: It's what allows them to invest in innovation.
8/ An important paper by Natalie Ram (forthcoming in @nulawreview) takes this argument seriously, and leaves it in shreds. She finds that commercial vendors of justice system tech *do not* need trade secret protection, in order to thrive and innovate. papers.ssrn.com/sol3/papers.cf…
9/ Short version: There are lots of ways to incentivize innovation, without secrecy. For example, patents combine economic incentives with public disclosure. And the landscape of options is much richer than trade secrets and patents.
10/ Sometimes, people speak as if the incentives for secrecy were extra-strong in criminal justice. That's what Harris, the maker of Stingray tools that surveil cell phones, claimed in 2010.
11/ In fact, however, the opposite is true: Rationales for secrecy are much weaker when criminal law is involved.
12/ And the other, non-secrecy methods of spurring innovation work *particularly well* in criminal justice, where government is the customer.
13/ These include "regulatory exclusivities," which many vendors of CJ tech already enjoy
15/ Approaches that require disclosure broadly, and early, have much to recommend them.
16/ (For clarity: my #AccountableAlgorithms coauthors & I have highlighted some limits to the value of source code disclosure, but we agree that code disclosure can be vital, including in courtroom settings where coding errors have constitutional stakes. papers.ssrn.com/sol3/papers.cf… )
17/ In sum, Natalie's paper makes a powerful case that the "commercial reasons" for criminal justice secrecy are nowhere near strong enough to keep source code secret. Read the paper! End of thread.
PS: That last screen shot was from Accountable Algorithms. Twitter has no edit button! Natalie's paper, again, is here. papers.ssrn.com/sol3/papers.cf…
