When designing a machine learning model, remember the "stretch pants" approach:
Don't waste time looking for pants that perfectly match your size. Instead, use large stretch pants that will shrink down to the right size.
What does this mean for your model?
Don't waste time looking for pants that perfectly match your size. Instead, use large stretch pants that will shrink down to the right size.
What does this mean for your model?
The "stretch pants" approach in machine learning:
Pick a model with more capacity than you need. Then, use regularization techniques to avoid overfitting.
You gotta thank Vincent Vanhoucke, a scientist at Google, for this analogy.
Pick a model with more capacity than you need. Then, use regularization techniques to avoid overfitting.
You gotta thank Vincent Vanhoucke, a scientist at Google, for this analogy.
One example:
Imagine designing a neural network, and you configure a hidden layer that's too small (not many neurons.)
The network may not preserve all the valuable information from the data: you don't have enough power to do it!
Realizing this is difficult.
Imagine designing a neural network, and you configure a hidden layer that's too small (not many neurons.)
The network may not preserve all the valuable information from the data: you don't have enough power to do it!
Realizing this is difficult.
If you start with a larger-than-you-may-need network, you can always regularize it back.
(You can also make it smaller when you have enough information to do so.)
As a recap:
1. Start big
2. Cut back
3. Regularize
(You can also make it smaller when you have enough information to do so.)
As a recap:
1. Start big
2. Cut back
3. Regularize
• • •
Missing some Tweet in this thread? You can try to
force a refresh
