🎉CutLER is accepted to #CVPR2023!
tl;dr: CutLER is an unsupervised object detector surpassing prev SOTA by 2.7x on 11 datasets across various domains, e.g. natural images, painting, sketch.
Codes/demos are released!
Developed during my internship at FAIR-Meta AI @MetaAI
1/n
Code: github.com/facebookresear…
HuggingFace demos (MaskCut): huggingface.co/spaces/faceboo…
HuggingFace demos (CutLER): huggingface.co/spaces/faceboo…
Colab demos: colab.research.google.com/drive/1NgEyFHv…
2/n
Demos results on 11 benchmarks spanning a variety of domains, including video frames, paintings, clip arts, complex scenes, etc.
3/n
Cut-and-Learn has two main stages: 1) generating pseudo-masks with our MaskCut without human annotations and 2) learning unsupervised detectors from pseudo-masks of unlabeled data.
4/n
Stage 1: The proposed MaskCut method can be used to provide segmentation masks for multiple instances of each image, leveraging a self-supervised pretrained DINO model.
5/n
MaskCut can produce high-quality segmentation masks for multiple objects!
Colab demo @GoogleColab: colab.research.google.com/drive/1X05lKL_…
HuggingFace demo @huggingface : huggingface.co/spaces/faceboo…
Replicate demo: replicate.com/cjwbw/cutler
6/n
Stage 2: After producing pseudo-masks with MaskCut, we then learn a detector using our loss-dropping strategy that is robust to objects missed by MaskCut. We further improve the model using multiple rounds of self-training.
7/n
Additional CutLER demos showcasing its ability to detect and segment small objects!
Try it out:
Colab demos @GoogleColab: colab.research.google.com/drive/1NgEyFHv…
HuggingFace demos @huggingface: huggingface.co/spaces/faceboo…
8/n
CutLER can also serve as a strong pretrained model for fully-supervised and semi-supervised object detection, surpassing MoCo-v2 by 7.3% APbox and 6.6% APmask on COCO when training with 5% labels.
9/n
Many thanks to my collaborators 🤗 Ishan @imisra_, Rohit @_rohitgirdhar_ and Stella, and the incredible internship experience at FAIR-Meta AI @MetaAI! Can't wait to be back next summer! 🎉🎉🎉
n/n
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.