1/ Interesting & troubling @drewharwell story. Even assuming that these AI screens of social media are unbiased and valid predictors (a major assumption), they're still deeply problematic. Analogizing it to a screening test shows why. washingtonpost.com/technology/201…
2/ As with any test, there's a sensitivity/specificity tradeoff. If the algorithm is set to be very specific, it will let more false negatives through (i.e., will be more likely to give problematic babysitters a pass). That will make customers unhappy.
3/ On the other hand, if it's set to be very sensitive, it will flag many babysitters who would be just fine as potentially problematic. That will hurt a lot of perfectly wonderful young people.
4/ Furthermore, the interpretation of the test depends on the prevalence of problematic babysitters in the pool of applicants. Unless the test is incredibly specific (but see #2, above), the positive predictive value of the test will be extremely poor.
5/ Said another way, the vast majority of applicants flagged as potentially problematic will be false positives--in fact, they will be just fine.
6/ To summarize: even without factoring in the inevitable problems with validity and bias, these tests are almost certainly a mess.
