Overfitting sucks.
Here are 7 ways you can deal with overfitting in Deep Learning neural networks.
π§΅π
Here are 7 ways you can deal with overfitting in Deep Learning neural networks.
π§΅π
A quick reminder:
When your model makes good predictions on the same data that was used to train it but shows poor results with data that hasn't seen before, we say that the model is overfitting.
The model in the picture is overfitting.
π
When your model makes good predictions on the same data that was used to train it but shows poor results with data that hasn't seen before, we say that the model is overfitting.
The model in the picture is overfitting.
π
1β£ Train your model on more data
The more data you feed the model, the more likely it will start generalizing (instead of memorizing the training set.)
Look at the relationship between dataset size and error.
(Unfortunately, sometimes there's no more data.)
π
The more data you feed the model, the more likely it will start generalizing (instead of memorizing the training set.)
Look at the relationship between dataset size and error.
(Unfortunately, sometimes there's no more data.)
π
2β£ Augment your dataset
You can automatically augment your dataset by transforming existing images in different ways to make the data more diverse.
Some examples:
β«οΈZoom in/out
β«οΈContrast changes
β«οΈHorizontal/vertical flips
β«οΈNoise addition
π
You can automatically augment your dataset by transforming existing images in different ways to make the data more diverse.
Some examples:
β«οΈZoom in/out
β«οΈContrast changes
β«οΈHorizontal/vertical flips
β«οΈNoise addition
π
3β£ Make your model simpler
You can:
β«οΈReduce the number of layers
β«οΈReduce the number of weights
The more complex your model is, the more capacity it has to memorize the dataset (hence, the easier it will overfit.)
Simplifying the model will force it to generalize.
π
You can:
β«οΈReduce the number of layers
β«οΈReduce the number of weights
The more complex your model is, the more capacity it has to memorize the dataset (hence, the easier it will overfit.)
Simplifying the model will force it to generalize.
π
4β£ Stop the learning process before overfitting
This is known as "Early Stopping."
Identify when overfitting starts happening and stop the learning process before it does.
Plotting the training and validation errors will give you what you need for this.
π
This is known as "Early Stopping."
Identify when overfitting starts happening and stop the learning process before it does.
Plotting the training and validation errors will give you what you need for this.
π
5β£ Standardize input data
Smaller weights can result in a model less prone to overfit.
Rescaling input data is a way to constraint these weights and keep them from increasing disproportionally.
π
Smaller weights can result in a model less prone to overfit.
Rescaling input data is a way to constraint these weights and keep them from increasing disproportionally.
π
6β£ Use Dropouts
Dropout is a regularization method that randomly ignores some of the outputs of a layer.
This simulates the process of training different neural networks with different architectures in parallel, which is a way to avoid overfitting.
machinelearningmastery.com/dropout-for-reβ¦
Dropout is a regularization method that randomly ignores some of the outputs of a layer.
This simulates the process of training different neural networks with different architectures in parallel, which is a way to avoid overfitting.
machinelearningmastery.com/dropout-for-reβ¦
7β£ L1 and L2 regularization
These refer to a technique that penalizes the loss function to keep the weights of the network constrained.
This means that the network is forced to generalize better because it can't grow the weights without limit.
π
These refer to a technique that penalizes the loss function to keep the weights of the network constrained.
This means that the network is forced to generalize better because it can't grow the weights without limit.
π
Is there anything else you use to prevent your models from overfitting?
β’ β’ β’
Missing some Tweet in this thread? You can try to
force a refresh
γ
