Share this page!

Jean de Nyandwi Profile picture
Twitter logo
20 Sep, 9 tweets, 2 min read
The key differences between shallow learning and deep learning models:

Shallow learning models:

◆ Most of them are simple and require less hyper-parametrization
◆ They need the features to be pre-extracted
◆ They are best suited for tabular datasets
◆ Their architectural changes are very limited.
◆ They don't require huge computation resources
◆ Their results are interpretable than deep learning models
◆ Because of the limit in their design change, there are little researches going on in these models.
Example of shallow learning models:

◆Linear and logistic regression
◆Support vector machines
◆Decision trees
◆Random forests
◆K-Nearest neighbors
Deep learning models:

◆ They are flexible in design. You can stack neural networks layers the way you like.
◆ They have many hyper-parameters and are hard to tune, but with tools like Keras Tuner, it's possible to get over it.

◆ They can extract features in data themselves
◆ They are best suited for complex problems. Often in problems that involve perception and language. Like object detection, image recognition, sentiment analysis, machine translation, etc...

◆ They require a huge amount of data and compute resources
◆ Their predictions are hard to interpret. Some say that interpreting them is nearly impossible.

◆ Because of their flexible design, there is a lot of space for researches.
Example of deep learning algorithms:

◆Densely connected neural networks
◆Convolutional neural networks
◆Recurrent neural networks
◆Transformers
Thank you for reading.

For more content like these, make sure to follow @Jeande_d.
For more about deep learning algorithms,

I wrote a long thread about them 👇

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Jean de Nyandwi

Jean de Nyandwi Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @Jeande_d

Jean de Nyandwi Profile picture
20 Sep
TensorFlow or PyTorch?

Forget about numbers. They are both great at what they do, which is putting machine learning codes together.
TensorFlow is most popular in industries, and PyTorch in research organizations/academics,

but the number of industries that use PyTorch and the number of researches made with TensorFlow have been all increasing.
If you are choosing what to learn for the first time, what is the best than the other does not really matter that much.

Focus on one, know its ins and outs, avoid going back and forth learning all of them, and let everybody else use their favorite tools.
Read 5 tweets
Jean de Nyandwi Profile picture
11 Sep
Popular deep learning architectures:

◆ Densely connected neural networks
◆ Convolutional neural networks
◆ Recurrent neural networks
◆ Transformers

Let's talk about these architectures and their suites of datasets in-depth 🧵
Machine learning is an experimentation science. An algorithm that was invented to process images can turn out to work well on texts too.

The next tweets are about the main neural network architectures and their suites of datasets.
1. Densely connected neural networks

Densely connected networks are made of stacks of layers that go from the input to the output.

Generally, networks are organized into layers. Each carry takes input data, processes it, and gives the output to the next layer.
Read 34 tweets
Jean de Nyandwi Profile picture
10 Sep
Neural networks are hard to train. The more they go deeper, the more they are likely to suffer from unstable gradients.

Gradients can either explode or vanish, and either of those can cause the network to give poor results.

A short thread on the neuralnets training issues
The vanishing gradients problem results in the network taking too long to train(learning will be very slow), and the exploding gradients cause the gradients to be very large.
Although those problems are nearly inevitable, the choice of activation function can reduce their effects.

Using ReLU activation in the first layers can help avoid vanishing gradients.

Careful weight initialization can also help, but ReLU is by far the good fix.
Read 4 tweets
Jean de Nyandwi Profile picture
6 Sep
Machine learning is the science of teaching the computer to do certain tasks, where instead of hardcoding it, we give it the data that contains what we want to achieve, and its job is to learn from such data to find the patterns that map what we want to achieve and provided data.
These patterns or (learned) rules can be used to make predictions on unseen data.
A machine learning model is nothing other than a mathematical function whose coefficient and intercept hold the best (or learned) values representing the provided data & what we want to achieve.

In ML terms, coefficients are weights, intercepts are biases.
Read 20 tweets
Jean de Nyandwi Profile picture
31 Aug
Everyone talks about big data but getting good data in a big amount is not always easy.

You can do much with small data as long as it is good.

A thread 🧵: Getting the most results with small data
The two notable techniques that can give huge results when working with small data are

◆ Data augmentation
◆ Transfer learning

Let's talk about them. We will use them in the context of images kind datasets but they can also be applied to other datasets such as texts.
1. Data augmentation

Data augmentation is the art of creating artificial (but realistic) data.

Not only does data augmentation expand the dataset,

but it also introduces some diversity in the training set (the reason why data augmentation is a cure for overfitting)
Read 12 tweets
Jean de Nyandwi Profile picture
27 Aug
Getting started with machine learning can be hard.

We are fortunate to have many & freely available learning resources, but most of them won't help because they skip the fundamentals or start with moonshots.

This is a thread on learning machine learning & structured resources.
1. Get excited first

The first step to learning a hard topic is to get excited.

Machine learning is a demanding field and it will take time to start understanding concepts & connecting things.
If you find it hard to understand what ML really is,

@lmoroney I/O 19 talk will get you excited. He introduces what machine learning really is from a programming perspective.



This talk never gets old to me.
Read 29 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

Like this author's thread?

Get emails when @Jeande_d has new unrolls!

Check out other threads from @Jeande_d!

Read all threads by @Jeande_d

:(