Tweet

Tivadar Danka

1 Mar, 13 tweets, 6 min read

You ask me so often for free online resources about deep learning that I decided to collect my favorite courses!

These topics interest you the most:

🟩 practical deep learning,
🟩 deep learning theory,
🟩 math resources to understand the two above.

Let's see them!

🧵 👇🏽

1️⃣ Practical deep learning.

If you want to take a deep dive straight into the field and want to start training your models right away, hands down the best course for you out there is Practical Deep Learning for Coders by fast.ai. (course.fast.ai)

@full_stack_dl

To move beyond training models and learn about tooling and infrastructure, IMO the best course for you is the Full Stack Deep Learning course by @full_stack_dl.

fall2019.fullstackdeeplearning.com

You can start these without any background in advanced mathematics. You'll even learn a lot on the way.

Both of these courses offer significant knowledge, and if you want to move more towards engineering, they cover all you need to know to start.

2️⃣ Deep learning theory.

If you want to see what is behind the curtain, my main recommendation is the second part of the fast.ai course: Deep Learning from the Foundations (course19.fast.ai/part2).

Another great source is the Stanford online course library.

Two especially awesome ones:

• CS231N Convolutional Neural Networks for Visual Recognition youtube.com/playlist?list=…

• CS224N Natural Language Processing with Deep Learning youtube.com/playlist?list=…

If you are looking for more classical machine learning (like decision trees or Gaussian mixture models), the channel of Mathematical Monk is hands down my favorite resource.

youtube.com/user/mathemati…

@3blue1brown

A gem in this subject is the 4 chapters long mini-series of @3blue1brown about the inner workings of neural networks. This is worth watching even if you don't need to understand all the fine details.

youtube.com/playlist?list=…

3️⃣ Mathematical foundations.

Math lies at the very foundations of deep learning. Although advanced math is not necessary to be an expert (especially if you are more into engineering instead of data science), it can elevate your understanding to the next level.

However, this is probably the most difficult subject to begin with.

There are three pillars of machine learning math:

🟩 linear algebra,
🟩 multivariate calculus,
🟩 and probability.

My two favorite online resources are the Khan Academy and MIT OpenCourseWare lectures.

Khan Academy is geared towards more beginners, while the MIT courses go into the very fine details.

If you are new to math, I recommend starting with the Khan Academy lectures.

Here are the links for the resources!

Linear algebra:
• khanacademy.org/math/linear-al…
• youtube.com/playlist?list=…

Multivariate calculus:
• khanacademy.org/math/multivari…
• youtube.com/playlist?list=…

Probability:
• khanacademy.org/math/statistic…
• youtube.com/playlist?list=…

What are your favorite online courses? Are there essential topics that these don't cover?

Share it with us!

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @TivadarDanka

Tivadar Danka

@TivadarDanka

2 Mar

Besides Kaggle, there are several other competition platforms.

You can use these to

• learn,
• test your skills,
• collaborate with awesome people,
• enhance your resume,
• and possibly earn money.

Take a look at these below, you'll definitely find them useful!

🧵 👇🏽

1. Numerai (numer.ai)

This is quite a special one, since it only contains a single competition.

However, its aims are big: Numerai wants to build the world's first open hedge fund

2. AIcrowd (aicrowd.com)

You can find all sort of competitions here on a wide spectrum, from applied problems to research.

Read 16 tweets

Tivadar Danka

@TivadarDanka

26 Feb

Have you ever thought about why neural networks are so powerful?

Why is it that no matter the task, you can find an architecture that knocks the problem out of the park?

One answer is that they can approximate any function with arbitrary precision!

Let's see how!

🧵 👇🏽

From a mathematical viewpoint, machine learning is function approximation.

If you are given data points 𝑥 with observations 𝑦, learning essentially means finding a function 𝑓 such that 𝑓(𝑥) approximates the given 𝑦-s as accurately as possible.

Approximation is a very natural idea in mathematics.

Let's see a simple example!

You probably know the exponential function well. Do you also know how to calculate it?

The definition itself doesn't really help you. Calculating the powers where 𝑥 is not an integer is tough.

Read 16 tweets

Tivadar Danka

@TivadarDanka

24 Feb

Conditional probability is one of the central concepts of statistics and probability theory.

Without a way to account for including prior information in our models, statistical models would be practically useless.

Let's see what conditional probability simply means!

If 𝐴 are 𝐵 are two events, they are not necessarily independent of each other.

This means that the occurrence of one can give information about the other.

When performing statistical modeling, this is frequently the case.

To illustrate, we will take a look at spam filters!

Suppose that you have 100 mails in your inbox.

40 is spam, 60 is not.

Based only on this information, if you receive a random letter, there is a 40% chance that it is spam.

This is not sufficient to build a decent model for spam detection.

Read 7 tweets

Tivadar Danka

@TivadarDanka

23 Feb

What makes it possible to train neural networks with gradient descent?

The fact that the loss function of a network is a differentiable function!

Differentiation can be hard to understand. However, it is an intuitive concept from physics.

💡 Let's see what it really is! 💡

Differentiation essentially describes a function's rate of change.

Let's see how!

Suppose that we have a tiny object moving along a straight line back and forth.

Its movement is fully described by its distance from the starting point, plotted against the time.

What is its average speed in its 10 seconds of travel time?

The average speed is simply defined as the ratio of distance and time.

However, it doesn't really describe the entire movement. As you can see, the speed is sometimes negative, sometimes positive.

Read 11 tweets

Tivadar Danka

@TivadarDanka

22 Feb

You can explain the Bayes formula in pure English.

Even without using any mathematical terminology.

Despite being overloaded with seemingly complex concepts, it conveys an important lesson about how observations change our beliefs about the world.

Let's take it apart!

Essentially, the Bayes formula describes how to update our models, given new information.

To understand why, we will look at a simple example with a twist: coin tossing with an unfair coin.

Let's suppose that we have a magical coin! It can come up with heads or tails when tossed, but not necessarily with equal probability.

The catch is, we don't know the exact probability. So, we have to perform some experiments and statistical estimation to find that out.

Read 14 tweets

Tivadar Danka

@TivadarDanka

21 Feb

It is the weekend now, so let's talk about something different, but still awesome and beautiful!

This image has been my desktop wallpaper for years.

Can you guess what is it?

This machine represents one of the most brilliant ideas I have seen. (Answer in the next tweet.)

This is the Wankel engine, a surprisingly innovative type of internal combustion engines.

Why is it so brilliant? In short, because it parallelizes the classical four-stage Otto cycle, all in one chamber!

To elaborate a bit, let's see how a four-stroke piston engine works!

The common four-stroke piston engine essentially has four stages:

1. Intake
2. Compression
3. Combustion
4. Exhaust

These happen in sequence inside a cylinder-shaped chamber, as shown below.

(Gifs and images in the thread are all from Wikipedia.)