Thread by @math_rachel

Is your machine learning solution creating more problems than it solves? @JasmineMcNealy on how in focusing narrowly on one problem, we may be missing many others @DataInstituteSF CADE Seminar Series

21 States Are Now Vetting Unemployment Claims With a ‘Risky’ Facial Recognition System:

"legitimate claimants have also been rejected by the company’s machine learning & facial recognition systems — leading to massive delays in life-sustaining funds"

onezero.medium.com/21-states-are-…

Central considerations when designing algorithms:
- *Who* is the prototype?
- Continuous evaluation & auditing
- We need to normalize STOPPING the use of a tool when harm is occurring. We don't need to keep using a tool just because we've already started. @JasmineMcNealy

We need to rethink data, not as an individual concern solved by individual consent, but as about relationships & networks.

So often technical systems are aimed at solving symptoms, not the root causes. Policy makers need to attempt solving the root issues first. -- @JasmineMcNealy

Do we really need to automate everything? Who is benefiting and who is being harmed?

Sometimes we don't need automation.

I really enjoyed this talk. Check out the rest of the spring data ethics @DataInstituteSF seminar line-up here:

usfblogs.usfca.edu/dataethics/202…

That was fast! The video is already up for Dr. McNealy's talk, "Looking Beyond the Prototype: How 'supposed to' makes way for harm"

@DataInstituteSF