Meticulous piece by @elize_lam pointing out the dangers of data science and machine learning approaches to profile and track 'risky families' combining data of children, parents and social context. A thread.. 1/10
Despite a large body of studies and red flags about the problematic effects of using data driven methods to predict child abuse, the decentralization of policy and budgeting to municipalities has motivated a plethora of local experiments as disguised austerity measures. 2/10
The Netherlands has a growing body of data sets with 'micro statistics' of citizens, maintained by the Central Bureau of Statistics (CBS), collected for 'statistical' purposes, but now increasingly used for such questionable predictive applications. 3/10
CBS has identified this problematic 'proliferation', in which data sets and algorithms seem to travel across governmental organizations with little supervision of who gets to see and work on what, and enacted some early measures to prevent misapplication, but more is needed. 4/10
While these efforts might have an intention to 'do good' or prevent harm, embedding algorithms in safety-critical contexts without proper validation is bound to cause new forms of harm and rights violations, as is now crystal clear (see our report: ainowinstitute.org/AI_Now_2019_Re…) 5/10
The truth is that the 'AI' many policy makers are in awe of typically has horrible accuracy, does not account for how humans interact with it, and will never be able to 'reason by itself' about what is ethical, legal or responsible. How intelligent is that really? 6/10
The way most of these pilots are organized is a disgrace not just in the eyes of scientists and engineers, it also further distances administrators from the structural issues that their citizens are dealing with. AI is a perfect 'context stripping machines' (@undersequoias) 7/10
If you are interested in an example, see this thread I wrote about the Rotterdam context (in Dutch): 8/10
Urgent interventions are needed to understand the full extent of this problem across policy domains, in order to set rules and boundaries and to decide what governance and enforcement mechanisms are needed to ensure these are not crossed. 9/10
The Dutch parliament will have to think creatively about what it means to promote innovation with digital technology. Pushing social problems to municipalities without funding, oversight or research to understand what works and doesn't is a recipe for disaster 10/10
Piece in Google Translate: translate.google.com/translate?sl=n…
