For many problems, a batch size of 32 works so well.
A batch size mostly affects training time. The larger the batch size, the faster the training.
The smaller, the slower training.
A batch size mostly affects training time. The larger the batch size, the faster the training.
The smaller, the slower training.
https://twitter.com/DataScienentist/status/1423014770777673729
The only issue with the large batch size is that it requires many steps per epoch to reach optimal performance.
And you need to have a large dataset in order to have enough steps per epoch.
With that said, 32 is a good default value to try at first.
And you need to have a large dataset in order to have enough steps per epoch.
With that said, 32 is a good default value to try at first.
Here are 2 great papers that you can use to learn more:
Practical Recommendations for Gradient-Based Training of Deep Architectures: arxiv.org/pdf/1206.5533.…
Practical Recommendations for Gradient-Based Training of Deep Architectures: arxiv.org/pdf/1206.5533.…
This paper suggests a different approach: Use a large batch size but start with a small learning rate and adjust accordingly...
Train longer, generalize better: closing the generalization gap in large batch training of neural networks
arxiv.org/pdf/1705.08741…
Train longer, generalize better: closing the generalization gap in large batch training of neural networks
arxiv.org/pdf/1705.08741…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
