When a machine learning system uses argmax to select outputs from a probability distribution — and most of them do — it's a clue that it might be biased. That's because argmax selects the "most probable" output, which may amplify tiny data biases into perfectly biased outputs.
Here's an exercise (with solution) I developed for my Fairness in ML course with @ang3linawang's help. It uses a toy model to show how bias amplification like the one in the "Men also like shopping" paper can arise through the use of argmax alone! drive.google.com/file/d/1baK_c4…
This graph is the punchline. α and β are parameters that describe correlations in the input and the graphs show correlations in the (multilabel) output. It should be terrifying from a scientific and engineering perspective even if there are no relevant fairness considerations! 
The nonmonotonicity & discontinuity mean that systems that use argmax to select outputs from a probability distribution can behave unintuitively and unpredictably. All bets are off if there is a distribution shift when the model is deployed. (There's always a distribution shift.)
Remember how Twitter's image cropping algorithm preferred lighter-skinned people? Twitter investigated and found that the use of argmax was one reason.
See Section 3.5 arxiv.org/pdf/2105.08667….
Not the only reason of course, but it's a bias amplifier.
See Section 3.5 arxiv.org/pdf/2105.08667….
Not the only reason of course, but it's a bias amplifier.
https://twitter.com/bascule/status/1307440596668182528
Remember gender biases in online translation? As long as argmax is used, these will probably persist. Even if the training corpus is congruent with a particular stereotype 51%-49%, the output will reflect the stereotype every time.
https://twitter.com/lauraolin/status/1371909490107428866
There's a subtle reason why argmax is a severe problem for multi-label classification, structured prediction, sequence prediction, etc., and much less serious for vanilla classification. This means it crops up all the time in computer vision, NLP, and recommender systems.
I don't know any effective technical fixes for the argmax issue. But a really good approach for handling model uncertainty is to show the user the possible outputs and ask them to choose. But this goes against the mantra of frictionless design and so it's very rarely deployed.
Generalizing a bit, a recurring theme in my course was that many fairness issues in online services can probably only be effectively addressed by **co-designing the user interface and the ML backend**. This of course violates core principles of sound engineering.
It's time for the fairness conversation to move beyond narrow questions of data and algorithms and grapple with thornier issues like the fact that design values and engineering best practices (which are in turn shaped by business models) often get in the way of mitigating biases.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
