Thread by @svpino

I've talked about Transfer Learning before.

In summary: you can reuse the knowledge from a different model to kick-start your new model.

Practically, this is how I make transfer learning happen: 🧵👇

First, I pick the model architecture I'll be transferring from.

There are hundreds of pre-trained models for TensorFlow (Check TensorFlow Hub.)

I spend most of my time working with images, and my go-to is usually ResNet with ImageNet weights.

↓ 2/10

I instantiate the model without its top layer and load the pre-trained weights into it.

To make sure I don't destroy those weights during training, I freeze those weights. Frozen weights won't change.

↓ 3/10

Freezing the imported weights is important.

If you don't do this, the weights will just serve as a sensible initialization, but the training process will destroy them.

↓ 4/10

You can also freeze part of the base model but leave a few layers open, so the training process readjusts the weights.

You'll need to explore whether this is best for your problem.

↓ 5/10

I then connect a few layers to the base model. These are the layers that will learn the specifics of my problem.

These layers won't be frozen.

↓ 6/10

At this point, I'm ready to train the model.

I usually go with a good, mid-of-the-road learning rate. My goal here is to get the open layers to learn their stuff.

↓ 7/10

Once the model is trained, and I'm happy with it, I try to fine-tune it to get as much juice from the orange.

To fine-tune the model:

▫️ Grab all the data
▫️ Unfreeze the base model layers
▫️ Set the learning rate to a very low value
▫️ Retrain the model

↓ 8/10

Here is more information about Transfer Learning, in case you are curious:

https://twitter.com/svpino/status/1372532791922069511?s=20

↓ 9/10

If you want to support my effort to put as much machine learning content out there as possible, please give me a follow and like and retweet this thread.

Your support is the fuel that keeps me going!

10/10