Thread by @lizjosullivan

Next up excited for @KLdivergence with a risk assessment tool analysis on “overbooking”. This is when someone is arrested on charges that are more serious than they should be. First off: there’s very little accountability so this is gonna be a hard problem to address. #FAT2020

Guess what? There is a lot of racial disparity in this. I’ll say... But again, this is hard to address because there’s no real “ground truth”. It’s complicated and we can attack the issue by looking at actual convictions vs. accusations.

So let’s look at the pre-trial risk assessment algos. One in particular in the USA is made of many models that output a score. Green = freedom. Red = you go to jail.

And, there are exceptions:

Also, exceptions the other way:

So let’s audit this mfer! Ran an experiment twice, that way we’re able to see if the predictions were correct, years later. Was this person actually convicted? Well... in 27% of the cases, people were accused of much greater crimes than they were convicted of, in the end.

Wow. That’s one in four. One in four people are arrested on charges that aren’t supported by enough facts to convict. We should spend more time working with judges so they are aware of this. YES!