Supervised ML methods (i.e. ERM) assume that train & test data are from the same distribution, & deteriorate when this assumption is broken.
To help, we introduce adaptive risk minimization (ARM):
arxiv.org/abs/2007.02931
With M Zhang, H Marklund @abhishekunique7 @svlevine
(1/6)
To help, we introduce adaptive risk minimization (ARM):
arxiv.org/abs/2007.02931
With M Zhang, H Marklund @abhishekunique7 @svlevine
(1/6)
Prior works on dIstributionally-robust optimization (DRO) aim to be _robust_ to distribution shift.
Group DRO aims for robustness to shifts in groups underlying the dataset. (e.g. see arxiv.org/abs/1611.02041)
(2/6)
Group DRO aims for robustness to shifts in groups underlying the dataset. (e.g. see arxiv.org/abs/1611.02041)
(2/6)
Recently, this paper showed promising results on group DRO with neural nets:
arxiv.org/abs/1911.08731
However, DRO methods often trade-off between robustness & test-time performance.
(3/6)
arxiv.org/abs/1911.08731
However, DRO methods often trade-off between robustness & test-time performance.
(3/6)
We aim to get *both* robustness and performance.
To do so, we introduce a new assumption to tackling group shift — that we get the unlabeled test data at once in a batch.
This resembles domain adaptation, except that the unlabeled test data is only available at test time.
(4/6)
To do so, we introduce a new assumption to tackling group shift — that we get the unlabeled test data at once in a batch.
This resembles domain adaptation, except that the unlabeled test data is only available at test time.
(4/6)
Key idea: Rather than trying to be robust, we train a meta-learning algorithm to *adapt* to group distribution shift with unlabeled data.
(5/6)
(5/6)
This generally leads to higher average-case and worst-case performance on both robustness benchmarks & a federated learning setting.
(6/6)
(6/6)
