A nice new fairness paper by Blum and Strangl: arxiv.org/pdf/1912.01094… They show that if there are two populations with the same base rate, but then data is biased either by undersampling positive examples from population B, or by corrupting positive labels in population B... 1/3
Then ERM subject to the constraint of equalizing true positive rates across groups recovers the optimal classifier on the original (unbiased) data distribution. Other fairness constraints (like also equalizing false positive rates, or asking for demographic parity) don't. 2/3
The really nice thing about this result is that unlike some other methods (like reweighting) for correcting biased data collection, this handles a relatively wide range of bias models in a detail free way: the ERM algorithm doesn't need to know the parsmeters of the bias model.
The result is also a little counter-intuitive. You might have thought that if the base rate is the same in the true distribution, but different in the data distribution (the construct and observed space, to use the terminology of @kdphd @geomblog @scheidegger)... 4/3
That the solution would be to ask for demographic parity --- i.e. enforce that the rate of positive classification is the same for each population, and that "error rate parity" conditions only make sense when you take the data as correct. But this paper shows the opposite. 5/3
This is another reason to like the "equal opportunity" style constraints proposed by @mrtz, Price, and Srebro in classification settings. 6/3
