[🧠 GAN paper summary 🧠] "Eyes Tell All" <- An interesting short paper on how to detect fake images generated by GANs (at least the current GAN methods like StyleGAN v2!).
They use a simple heuristic: 1/
They use a simple heuristic: 1/
1. Crop the face and then eyes (they used dlib for this)
2. Segment out the pupils -> you get the predicted mask
3. Fit an ellipse -> you get the "GT" mask
4. Find BIoU between the 2) and 3)
2/
2. Segment out the pupils -> you get the predicted mask
3. Fit an ellipse -> you get the "GT" mask
4. Find BIoU between the 2) and 3)
2/
(BIoU is just a generalization of IoU where we focus only on the boundary pixels of the mask instead of the whole mask)
5. If BIoU is "too low" that means that the predicted pupil is deviating from the elliptical shape and thus the eyes i.e. the face is fake/generated!
3/
5. If BIoU is "too low" that means that the predicted pupil is deviating from the elliptical shape and thus the eyes i.e. the face is fake/generated!
3/
Obviously, there are some cons here as well - some real people do have irregular iris shape (i.e. those irises deviate from elliptical shape) and thus the method would produce false positives for real humans that have certain eye conditions.
Paper: arxiv.org/abs/2109.00162
4/
Paper: arxiv.org/abs/2109.00162
4/
It's an interesting topic in general that will become increasingly important - how can we make sure that the media we're presented with is genuine?
IMHO, in the long run, these CV-based approaches are futile and we'll have to resort to some kind of digital signatures.
5/
IMHO, in the long run, these CV-based approaches are futile and we'll have to resort to some kind of digital signatures.
5/
But for the time being, if you feel paranoid about an image, you can check whether the iris is irregular to determine whether the image is fake or not (at least until StyleGAN v3 comes out haha) #gans
6/
6/
• • •
Missing some Tweet in this thread? You can try to
force a refresh
