Tivadar Danka Profile picture
Jun 30 16 tweets 5 min read Read on X
In calculus, going from a single variable to millions of variables is hard.

Understanding the three main types of functions helps make sense of multivariable calculus.

Surprisingly, they share a deep connection. Let's see why! Image
In general, a function assigns elements of one set to another.

This is too abstract for most engineering applications. Let's zoom in a little! Image
As our measurements are often real numbers, we prefer functions that operate on real vectors or scalars.

There are three categories:

1. vector-scalar,
2. vector-vector,
3. and scalar-vector. Image
When speaking about multivariable calculus, vector-scalar functions come to mind first.

Instead of a graph (like their single-variable counterparts), they define surfaces. Image
You can think about a vector-scalar function as a topographic map.

(Image source: Wikipedia, ) en.wikipedia.org/wiki/Terrain_c…Image
Although often not denoted, the argument of a vector-scalar function is always a vector.

Most frequently, we write out the components - a.k.a. the variables - explicitly. Image
Want a practical example of a vector-scalar function?

The loss of a predictive model maps the vector of parameters to a single scalar.

Below, you can see the mean-squared error of a simple linear regression model. Image
Next up, we have the vector-vector functions.

You can imagine them as a force field, putting a vector to each point. Image
The most important example of vector-vector functions is the gradient.

We call this a gradient field. Image
Let's visualize an example!

This is how the vector field given by the gradient of f(x, y) = x² + y² looks. Image
It is important to note that not all vector-vector functions are gradient fields!

For instance, f(x, y) = (x - xy, xy - y) cannot be a gradient.

Can you figure out the reason why? (Hint: take a look at the partial derivatives of f(x, y).) Image
Next up, we have scalar-vector functions, that is, curves.

Think about the scalar-vector function f(t) as the trajectory of a particle at time t.

Technically, there is only a single variable involved. Yet, curves play an essential role in multivariable calculus. Image
Remember how vector-vector functions define force fields?

Scalar-vector functions describe the trajectories of particles moving through them. Image
Gradient descent connects all of this.

In essence, gradient descent

1. takes the surface of the loss function,
2. computes the vector field given by the gradient,
3. and finds the trajectories given by the gradient vector field by a discrete approximation.
This is just the tip of the iceberg.

Multivariable calculus is one of the most powerful tools in machine learning, helping us to optimize functions in millions of variables.

That is quite a feat.
This explanation is directly from my new Mathematics of Machine Learning book.

I have packed 20 years of math studies into 700 pages full of intuitive and application-oriented lessons, the ultimate learning resource for you.

Get it now: amazon.com/Mathematics-Ma…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Tivadar Danka

Tivadar Danka Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @TivadarDanka

Jun 30
Neural networks are stunningly powerful.

This is old news: deep learning is state-of-the-art in many fields, like computer vision and natural language processing. (But not everywhere.)

Why are neural networks so effective? I'll explain. Image
First, let's formulate the classical supervised learning task!

Suppose that we have a dataset D, where xₖ is a data point and yₖ is the ground truth. Image
The task is simply to find a function g(x) for which

• g(xₖ) is approximately yₖ,
• and g(x) is computationally feasible.

To achieve this, we fix a parametrized family of functions. For instance, linear regression uses this function family: Image
Read 19 tweets
Jun 28
One major reason why mathematics is considered difficult: proofs.

Reading and writing proofs are hard, but you cannot get away without them. The best way to learn is to do.

So, let's deconstruct the proof of the most famous mathematical result: the Pythagorean theorem. Image
Here it is in its full glory.

Theorem. (The Pythagorean theorem.) Let ABC be a right triangle, let a and b be the length of its two legs, and let c be the length of its hypotenuse.

Then a² + b² = c². Image
Now, the proof. Mathematical proofs often feel like pulling a rabbit out of a hat. I’ll go a bit overboard and start by pulling out two rabbits.

The first rabbit. Take a look at the following picture.

The depicted square’s side is a + b long, so its area is (a + b)². Image
Read 19 tweets
Jun 26
Problem-solving is at least 50% of every job in tech and science.

Mastering problem-solving will make your technical skill level shoot up like a hockey stick. Yet, we are rarely taught how to do so.

Here are my favorite techniques that'll loosen even the most complex knots: Image
0. Is the problem solved yet?

The simplest way to solve a problem is to look for the solution elsewhere. This is not cheating; this is pragmatism. (Except if it is a practice problem. Then, it is cheating.)
When your objective is to move fast, this should be the first thing you attempt.

This is the reason why Stack Overflow (and its likes) are the best friends of every programmer.
Read 18 tweets
Jun 25
What you see below is one of the most beautiful formulas in mathematics.

A single equation, establishing a relation between 𝑒, π, the imaginary number, and 1. It is mind-blowing.

This is what's behind the sorcery: Image
First, let's go back to square one: differentiation.

The derivative of a function at a given point describes the slope of its tangent plane. Image
By definition, the derivative is the limit of difference quotients: slopes of line segments that get closer and closer to the tangent.

These quantities are called "difference quotients". Image
Read 20 tweets
Jun 24
"Probability is the logic of science."

There is a deep truth behind this conventional wisdom: probability is the mathematical extension of logic, augmenting our reasoning toolkit with the concept of uncertainty.

In-depth exploration of probabilistic thinking incoming. Image
Our journey ahead has three stops:

1. an introduction to mathematical logic,
2. a touch of elementary set theory,
3. and finally, understanding probabilistic thinking.

First things first: mathematical logic.
In logic, we work with propositions.

A proposition is a statement that is either true or false, like

• "it's raining outside",
• "the sidewalk is wet".

These are often abbreviated as variables, such as A = "it's raining outside".
Read 29 tweets
Jun 23
Understanding graph theory will seriously enhance your engineering skills; you must absolutely be familiar with them.

Here's a graph theory quickstart, in collaboration with @alepiad.

Read on: Image
What do the internet, your brain, the entire list of people you’ve ever met, and the city you live in have in common?

These are all radically different concepts, but they share a common trait.

They are all networks that establish relationships between objects. Image
As distinct as these things seem to be, they share common properties.

For example, the meaning of “distance” is different for

• Social networks
• Physical networks
• Information networks

But in all cases, there is a sense in which some objects are “close” or “far”. Image
Read 15 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(