"Probability is the logic of science."

There is a deep truth behind this conventional wisdom: probability is the mathematical extension of logic, augmenting our reasoning toolkit with the concept of uncertainty.

In-depth exploration of probabilistic thinking incoming. Our journey ahead has three stops:

1. an introduction to mathematical logic,
2. a touch of elementary set theory,
3. and finally, understanding probabilistic thinking.

First things first: mathematical logic.

Conditional probability is the single most important concept in statistics.

Why? Because without accounting for prior information, predictive models are useless.

Here is what conditional probability is, and why it is essential. Conditional probability allows us to update our models by incorporating new observations.

By definition, P(B | A) describes the probability of an event B, given that A has occurred.

How to build a good understanding of math for machine learning?

I get this question a lot, so I decided to make a complete roadmap for you. In essence, three fields make this up: calculus, linear algebra, and probability theory.

Let's take a quick look at them! 1. Linear algebra.

In machine learning, data is represented by vectors. Essentially, training a learning algorithm is finding more descriptive representations of data through a series of transformations.

Linear algebra is the study of vector spaces and their transformations.

Understanding graph theory will seriously enhance your engineering skills; you must absolutely be familiar with them.

Here's a graph theory quickstart, in collaboration with Alejandro Piad Morffis.

Read on: What do the internet, your brain, the entire list of people you’ve ever met, and the city you live in have in common?

These are all radically different concepts, but they share a common trait.

They are all networks that establish relationships between objects.

Neural networks are stunningly powerful.

This is old news: deep learning is state-of-the-art in many fields, like computer vision and natural language processing. (But not everywhere.)

Why are neural networks so effective? I'll explain. First, let's formulate the classical supervised learning task!

Suppose that we have a dataset D, where xₖ is a data point and yₖ is the ground truth.

A question we never ask:

"How large that number in the Law of Large Numbers is?"

Sometimes, a thousand samples are large enough. Sometimes, even ten million samples fall short.

How do we know? I'll explain. First things first: the law of large numbers (LLN).

Roughly speaking, it states that the averages of independent, identically distributed samples converge to the expected value, given that the number of samples grows to infinity.

We are going to dig deeper.

With the power of mathematical induction, I'll prove that everyone has the same eye color.

Don't believe me? Read on.

(And see if you can spot the sleight of hand.) To formalize the problem, define the proposition Aₙ by

Aₙ = "in a set of n people, everyone has the same eye color".

If n equals the human population of planet Earth, we get the original statement. We'll prove that Aₙ is true via induction.

The single biggest argument about statistics: is probability frequentist or Bayesian? It's neither, and I'll explain why.

Buckle up. Deep-dive explanation incoming. First, let's look at what is probability.

Probability quantitatively measures the likelihood of events, like rolling six with a dice. It's a number between zero and one. This is independent of interpretation; it’s a rule set in stone.

The Japanese multiplication method makes everybody feel "I wish they taught math like this in school."

It's not just a cute visual tool: it illuminates how and why long multiplication works.

Here is the full story. First, the Japanese multiplication method.

The first operand (21 in our case) is represented by two groups of lines: two lines in the first (1st digit), and one in the second (2nd digit).

One group for each digit.

One major reason why mathematics is considered difficult: proofs.

Reading and writing proofs are hard, but you cannot get away without them. The best way to learn is to do.

So, let's deconstruct the proof of the most famous mathematical result: the Pythagorean theorem. Here it is in its full glory.

Theorem. (The Pythagorean theorem.) Let ABC be a right triangle, let a and b be the length of its two legs, and let c be the length of its hypotenuse.

Then a² + b² = c².

Problem-solving is at least 50% of every job in tech and science.

Mastering problem-solving will make your technical skill level shoot up like a hockey stick. Yet, we are rarely taught how to do so.

Here are my favorite techniques that'll loosen even the most complex knots: 0. Is the problem solved yet?

The simplest way to solve a problem is to look for the solution elsewhere. This is not cheating; this is pragmatism. (Except if it is a practice problem. Then, it is cheating.)

Yesterday, I posted the following puzzle.

There is a 80 m long cable, strung out between two 50 m tall poles. The bottom of the hanging cable is 10 m above ground. How far are the two poles from each other?

Here is the solution: This problem is a typical instance of missing the forest for the trees.

If you gave it any thought, you probably attempted to apply fundamental physics to model the hanging cable, then calculate the distance using a complex system of equations.

This is not needed at all.

I've been studying math for 50% of my life.

The single most common question I get: why should I study mathematics as a ____? So, I have collected my thoughts for you.

Here are the most important things that math taught me: 1. The language of thinking.

Contrary to popular belief, math is not (only) about numbers. It's about abstraction and reasoning. This requires clear and concise thinking.

Thus, you can pick up an advanced thinking toolkit even from basic math.

Summing numbers is more exciting than you think.

For instance, summing the same alternating sequence of 1-s and (-1)-s can either be zero or one, depending on how we group the terms. What's wrong?

I'll explain. Enter the beautiful world of infinite series. Let’s go back to square one: the sum of infinitely many terms is called an infinite series. (Or series in short.)

Infinite series form the foundations of mathematics.

Matrices + the Gram-Schmidt process = magic.

This magic is called the QR decomposition, and it's behind the famous eigenvalue-finding QR algorithm.

Here is how it works. In essence, the QR decomposition factors an arbitrary matrix into the product of an orthogonal and an upper triangular matrix.

(We’ll illustrate everything with the 3 x 3 case, but everything works as is in general as well.)

I asked Salvador Dali to imagine some of the most beautiful mathematical theorems in his breathtaking paintings. (Or maybe it was Midjourney, in the style of Dali.)

Here are the results:

1. "The set of real numbers is uncountably infinite." 2. The Baire category theorem: "In a complete metric space, the intersection of countably many open dense sets remains dense."

I described some of the most beautiful and famous mathematical theorems to Midjourney.

Here is how it imagined them:

1. "The set of real numbers is uncountably infinite." 2. The Baire category theorem: "In a complete metric space, the intersection of countably many dense sets remains dense."

The Gram-Schmidt process is one of the most important algorithms in linear algebra.

Its task is simple: orthogonalizing vector sets.
Its applications are endless: matrix decompositions, eigenvalue problems, numerical linear algebra...

This is how it works: The problem is simple. We are given a set of basis vectors

a₁, a₂, …, aₙ,

and we want to turn them into an orthogonal basis

q₁, q₂, …, qₙ,

such that each qᵢ-s represent the same information as aᵢ.

The Gram-Schmidt process is one of the most important algorithms in linear algebra.

Its task is simple: orthogonalizing vector sets.
Its applications are endless: matrix decompositions, eigenvalue problems, numerical linear algebra...

This is how it works: The problem is simple. We are given a set of basis vectors

a₁, a₂, …, aₙ,

and we want to turn them into an orthogonal basis

q₁, q₂, …, qₙ,

such that each qᵢ-s represent the same information as aᵢ.

Here is a probabilistic puzzle.

Feedex and Acme are two delivery companies. Feedex trains are 80% on time, while only 40% of its trucks are.

However, Acme's trains are 100% on time, and 60% of its trucks are as well.

Yet, Feedex is more reliable! Why? This lesson is brought to you @brilliantorg's Introduction to Probability course. Their interactive, first-principles approach will make sure you understand and retain the things you learn there.

Since I'm partnering with them, I have a special offer for you later.

Let's go!

In machine learning, we take gradient descent for granted. We rarely question why it works.

What's usually told is the mountain-climbing analogue: to find the valley, step towards the steepest descent.

But why does this work so well? Read on. Our journey is leading through

• differentiation, as the rate of change,
• the basics of differential equations,
• and equilibrium states.

Buckle up! Deep dive into the beautiful world of dynamical systems incoming. (Full post link at the end.)