Many face recognition datasets have been taken down due to ethical concerns. In ongoing research, we found that this doesn't achieve much. For example, the DukeMTMC dataset of videos was used in 135 papers published *after* it was taken down in June 2019. freedom-to-tinker.com/2020/10/21/fac…
A major challenge comes from derived datasets. In particular, the DukeMTMC-ReID dataset is a popular dataset used for person re-identification and continues to be free for anyone to download. 116 of 135 papers that use DukeMTMC after its takedown actually use a derived dataset.
This is a widespread problem. MS-Celeb was removed due to criticism but lives on through MS1M-IBUG, MS1M-ArcFace, MS1M-RetinaFace… all still public. The original dataset is also available via Academic Torrents. One popular dataset, LFW, has spawned at least 14 derivatives.
Regulating research ethics is hard, and the machine learning community has only recently started thinking seriously about it. So it's important to empirically test whether current ways of regulating ethics are working. Our work suggests there's a long way to go.
The key challenge in our research was figuring out which papers actually use a dataset as opposed to merely mentioning or citing the dataset. We are currently writing up a paper where we describe how we achieve this and present many more findings.
Finally, an important caveat: we are able to study the use of dubious facial recognition datasets *for research* by looking at the papers that ultimately result. There is a whole world of questionable uses in industry and by governments, but we have no window into it.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
