Transfer Learning.
It sounds fancy because it is.
This is a thread about one of the most powerful tools that make possible that knuckleheads like me achieve state-of-the-art Deep Learning results on our laptops.
🧵👇
It sounds fancy because it is.
This is a thread about one of the most powerful tools that make possible that knuckleheads like me achieve state-of-the-art Deep Learning results on our laptops.
🧵👇
Deep Learning is all about "Deep" Neural Networks.
"Deep" means a lot of complexity. You can translate this to "We Need Very Complex Neural Networks." See the attached example.
The more complex a network is, the slower it is to train, and the more data we need to train it.
👇
"Deep" means a lot of complexity. You can translate this to "We Need Very Complex Neural Networks." See the attached example.
The more complex a network is, the slower it is to train, and the more data we need to train it.
👇
To get state-of-the-art results when classifying images, we can use a network like ResNet50, for example.
It takes around 14 days to train this network with the "imagenet" dataset (1,300,000+ images.)
14 days!
That's assuming that you have a decent (very expensive) GPU.
👇
It takes around 14 days to train this network with the "imagenet" dataset (1,300,000+ images.)
14 days!
That's assuming that you have a decent (very expensive) GPU.
👇
This, of course, is out of reach for people like you and me. We wouldn't be able to afford the cost of constantly experimenting with deep networks.
Cost and time are both ridiculous.
Thankfully, we don't have to. We have "Transfer Learning."
👇
Cost and time are both ridiculous.
Thankfully, we don't have to. We have "Transfer Learning."
👇
Let's get the boring definition out of the way:
▫️Transfer learning is a method where we can reuse a model that was developed for one task as the starting point for another task.
What does this mean?
We don't need to train our huge deep network from scratch!
👇
▫️Transfer learning is a method where we can reuse a model that was developed for one task as the starting point for another task.
What does this mean?
We don't need to train our huge deep network from scratch!
👇
Imagine we want to build a model to recognize a few animal species.
To get decent results, we'd need to train that model with thousands (millions) of images. Just think about all the information that's packed on an image! There's no way to learn without many, many images.
👇
To get decent results, we'd need to train that model with thousands (millions) of images. Just think about all the information that's packed on an image! There's no way to learn without many, many images.
👇
Thanks to Transfer Learning we don't have to!
ResNet50 was trained on 1.3M images, right? These images are not only animals, but they are a good representation of the world.
After training, ResNet50 knows about shapes, colors, shadows, lines, borders, etc.
👇
ResNet50 was trained on 1.3M images, right? These images are not only animals, but they are a good representation of the world.
After training, ResNet50 knows about shapes, colors, shadows, lines, borders, etc.
👇
All of that information is already encoded in that model and we don't need to learn it again.
Instead, we can use that model, modify its output, and train a portion of it to learn the specific animal species from our dataset.
We will be "inheriting" all its knowledge.
👇
Instead, we can use that model, modify its output, and train a portion of it to learn the specific animal species from our dataset.
We will be "inheriting" all its knowledge.
👇
This concept is extremely powerful.
Today, we have big organizations training these base models with huge datasets, and the rest of the industry can simply transfer all of that knowledge and focus on more specific problems.
We don't need 14 days to train anymore!
👇
Today, we have big organizations training these base models with huge datasets, and the rest of the industry can simply transfer all of that knowledge and focus on more specific problems.
We don't need 14 days to train anymore!
👇
Throughout this thread, I used ResNet50 as an example. Here are more architectures:
▫️Inception
▫️VGG
▫️NasNet
▫️Xception
▫️MobileNet
▫️EfficientNet
▫️DenseNet
I'm curious to hear about your experience. How are you taking advantage of Transfer Learning in your work?
▫️Inception
▫️VGG
▫️NasNet
▫️Xception
▫️MobileNet
▫️EfficientNet
▫️DenseNet
I'm curious to hear about your experience. How are you taking advantage of Transfer Learning in your work?
• • •
Missing some Tweet in this thread? You can try to
force a refresh
