,
5 tweets,
2 min read
Read on Twitter
Excited to share our new work with @dbharathbhushan, @nicolas_courty & more: Wasserstein Adversarial Training for label noise arxiv.org/abs/1904.03936…. We extended the concept of virtual adversarial training with a Wasserstein loss to control the loss evolution between classes. 
As shown in the previous figure, from the ground cost, one can control the loss evolution between classes with a Wasserstein loss. This is not possible with KL which is isotropic between classes. This control allows us to influence the classifier's decision boundary.
We can take advantage of it for label noise: between similar classes (red & orange classes) we want to have a complex boundary and set a high cost. And between non similar classes (orange/red & black) we want to have a simple boundary as it is probably noise and set a small cost.
With this approach, we can control which labels can break a certain local uniformity and which can't. To get the ground cost, we elegantly designed it from the word2vec framework. We designed it from the L2 norm between the word2vec class representations.
For experiments, we considered asymmetric noise (20% and 40%) on 4 datasets (Fashion MNIST, CIFAR10-100 and a remote sensing dataset). WAT is able to achieve new state of the art results for these benchmarks.
