Thread by @Jeande_d

Everyone talks about big data but getting good data in a big amount is not always easy.

You can do much with small data as long as it is good.

A thread 🧵: Getting the most results with small data

The two notable techniques that can give huge results when working with small data are

◆ Data augmentation
◆ Transfer learning

Let's talk about them. We will use them in the context of images kind datasets but they can also be applied to other datasets such as texts.

1. Data augmentation

Data augmentation is the art of creating artificial (but realistic) data.

Not only does data augmentation expand the dataset,

but it also introduces some diversity in the training set (the reason why data augmentation is a cure for overfitting)

Take an example: Image augmentation

Creating new images might mean:

◆Flipping the existing images
◆Cropping them
◆Changing their color and contrast and
◆Adding the noise to the existing images

Image credit: Alfonso Escalante

In most cases, data augmentation will improve the results.

It will only hurt the results if the added data are not realistic or were not needed.

More about data augmentation 👇🧵

https://twitter.com/Jeande_d/status/1422882083131826179?s=20

2. Transfer learning

There is a popular notion that says that "it's not a good idea to train a deep neural network from scratch" (CC: @aureliengeron, Hands-On ML)

Transfer learning is what makes such a notion a real thing.

Instead of training a neural network from scratch,

you can just borrow some parts of a neural network that is already trained (pre-trained models) and tweak it a little bit based on your problem.

Transfer learning can give great results on small datasets because you are taking advantage of pre-trained models,

but also it can save computation power and time.

More about transfer learning:

https://twitter.com/Jeande_d/status/1415987689715867649?s=20

The combination of data augmentation and transfer learning plays it well, and you can always try them.

I meant trying because machine learning is experimentation science. There is no single technique/model that is guaranteed to work on all possible problems.

This is the end!

To summarize:

When working with small data, data augmentation and transfer learning can boost the results.

Data augmentation expands the dataset, and transfer learning lets you use the models that someone else trained on the big dataset.

One more thing:

Even if you have enough data to solve a given problem, data augmentation is still worth trying.

Remember: it introduces diversity in the training set, and that is always a good thing. It is a cure for overfitting.

Thanks for reading!

If you have found this thread helpful, share it with anyone who you think would like to learn more about those techniques.

I also appreciate retweets. It is certainly the best way to help the post reach many people.

Follow @Jeande_d for more ML ideas!