Sunday Morning Reading Thread ☕️
- Self-Supervised Learning: The Dark Matter of Intelligence 🧠
- SEER ⚙️
- Multimodal Neurons 👁️📚
- Do Transformer Modifications Transfer? ⚔️
- Ultra Data-Efficient GAN 🤯
Quote from each below: 👇
- Self-Supervised Learning: The Dark Matter of Intelligence 🧠
- SEER ⚙️
- Multimodal Neurons 👁️📚
- Do Transformer Modifications Transfer? ⚔️
- Ultra Data-Efficient GAN 🤯
Quote from each below: 👇
Self-Supervised Learning: The Dark Matter of Intelligence 🧠
"As babies, we learn how the world works largely by observation. We form generalized predictive models about objects in the world by learning concepts such as object permanence and gravity"
ai.facebook.com/blog/self-supe…
"As babies, we learn how the world works largely by observation. We form generalized predictive models about objects in the world by learning concepts such as object permanence and gravity"
ai.facebook.com/blog/self-supe…
Self-Supervised Pretraining of Visual Features in the Wild ⚙️
"a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2% top-1 accuracy, surpassing the best self-supervised pretrained model by 1%"
arxiv.org/abs/2103.01988
"a RegNetY with 1.3B parameters trained on 1B random images with 512 GPUs achieves 84.2% top-1 accuracy, surpassing the best self-supervised pretrained model by 1%"
arxiv.org/abs/2103.01988
Multimodal Neurons 👁️📚
"We've discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually."
openai.com/blog/multimoda…
"We've discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually."
openai.com/blog/multimoda…
Do Transformer modifications Transfer? ⚔️
"The main goal of this paper is to try to determine why most modifications proposed to the Transformer have not seen widespread adoption."
arxiv.org/pdf/2102.11972…
"The main goal of this paper is to try to determine why most modifications proposed to the Transformer have not seen widespread adoption."
arxiv.org/pdf/2102.11972…
Ultra Data-Efficient GAN Training 🤯
"We decompose the data-hungry GAN training into two sequential sub-problems: (i) identifying the lottery ticket form the original GAN; then (ii) training the found sparse subnetwork with aggressive data and feature augmentations"
Paper below
"We decompose the data-hungry GAN training into two sequential sub-problems: (i) identifying the lottery ticket form the original GAN; then (ii) training the found sparse subnetwork with aggressive data and feature augmentations"
Paper below
• • •
Missing some Tweet in this thread? You can try to
force a refresh
