,
3 tweets,
2 min read
Read on Twitter
Assessing fairness of predictive risk scores requires us to think beyond binary classification. In arxiv.org/abs/1902.05826 we consider (un)fairness in bipartite ranking, where a natural metric, xAUC, arises for diagnosing disparities in risk scores algo @angelamczhou #NeurIPS2019
@angelamczhou Scores can have biases even when algo gives no clear binary decision (algos rarely make final decisions in civics). True story: Equivant (then Northpointe) pointed to overlapping ROCs to defend its COMPAS recidivism score. But xROC curves paint a different, more racist picture.
@angelamczhou The xROCs show that COMPAS is essentially random guessing when choosing whose riskier between a black non-recidivator and a white recidivator, but it makes damn sure to absolve white non-recidivators when compared to black recidivators. (Difference in probs = xAUC disparity) 
