Tivadar Danka Profile picture
Mar 3, 2022 13 tweets 5 min read Read on X
There is more than one way to think about matrix multiplication.

By definition, it is not easy to understand. However, there are multiple ways of looking at it, each one revealing invaluable insights.

Let's take a look at them!

↓ A thread. ↓
First, let's unravel the definition and visualize what happens.

For instance, the element in the 2nd row and 1st column of the product matrix is created from the 2nd row of the left and 1st column of the right matrices by summing their elementwise product.
To move beyond the definition, let's introduce some notations.

A matrix is built from rows and vectors. These can be viewed as individual vectors.

You can think of them as a horizontal stack of column vectors or a vertical stack of row vectors.
Let's start by multiplying a matrix and a row vector.

By writing out the definition, it turns out that the product is just a linear combination of the columns, where the coefficients are determined by the vector we are multiplying with!
Taking this one step further, we can stack another vector.

This way, we can see that the product of an (n x n) and an (n x 2) matrix equals the product of the left matrix and the columns of the right matrix, horizontally stacked.
Applying the same logic, we can finally see that the product matrix is nothing else than the left matrix times the columns of the right matrix, horizontally stacked.

This is an extremely powerful way of thinking about matrix multiplication.
We can also get the product as vertically stacked row vectors by switching our viewpoint a bit.
There is another interpretation of matrix multiplication.

Let's rewind and go back to the beginning, studying the product of a matrix 𝐴 and a column vector 𝑥.

Do the sums in the result look familiar?
These sums are just the dot product of the row vectors of 𝐴, taken with the column vector 𝑥!
In general, the product of 𝐴 and 𝐵 is simply the dot products of row vectors from 𝐴 and column vectors from 𝐵!
To sum up, we have three interpretations: matrix multiplication as

1. vertically stacking row vectors,
2. horizontally stacking column vectors,
3. and as dot products of row vectors with column vectors.

When studying matrices, each of them is immensely useful.
Having a deep understanding of math will make you a better engineer. I want to help you with this, so I am writing a comprehensive book about the subject.

If you are interested in the details and beauties of mathematics, check out the early access!

tivadardanka.com/book
If you have enjoyed this thread, consider giving it a retweet and following me!

I regularly post deep-dive explanations about seemingly complex concepts from mathematics and machine learning.

Mathematics is beautiful, and I want to show this to you.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Tivadar Danka

Tivadar Danka Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @TivadarDanka

Oct 15
I have spent at least 50% of my life studying, practicing, and teaching mathematics.

The most common misconceptions I encounter:

• Mathematics is useless
• You must be good with numbers
• You must be talented to do math

These are all wrong. Here's what math is really about: Image
Let's start with a story.

There’s a reason why the best ideas come during showers or walks. They allow the mind to wander freely, unchained from the restraints of focus.

One particular example is graph theory, born from the regular daily walks of the legendary Leonhard Euler.
Here is the map of Königsberg (now known as Kaliningrad, Russia), where these famous walks took place.

This part of the city is interrupted by several rivers and bridges.

(I cheated a little and drew the bridges that were there in Euler's time, but not now). Image
Read 15 tweets
Oct 14
In machine learning, we use the dot product every day.

However, its definition is far from revealing. For instance, what does it have to do with similarity?

There is a beautiful geometric explanation behind: Image
By definition, the dot product (or inner product) of two vectors is defined by the sum of coordinate products. Image
To peek behind the curtain, there are three key properties that we have to understand.

First, the dot product is linear in both variables. This property is called bilinearity. Image
Read 15 tweets
Oct 13
Matrix factorizations are the pinnacle results of linear algebra.

From theory to applications, they are behind many theorems, algorithms, and methods. However, it is easy to get lost in the vast jungle of decompositions.

This is how to make sense of them. Image
We are going to study three matrix factorizations:

1. the LU decomposition,
2. the QR decomposition,
3. and the Singular Value Decomposition (SVD).

First, we'll take a look at LU.
1. The LU decomposition.

Let's start at the very beginning: linear equation systems.

Linear equations are surprisingly effective in modeling real-life phenomena: economic processes, biochemical systems, etc. Image
Read 18 tweets
Oct 11
Behold one of the mightiest tools in mathematics: the camel principle.

I am dead serious. Deep down, this tiny rule is the cog in many methods. Ones that you use every day.

Here is what it is, how it works, and why it is essential: Image
First, the story:

The old Arab passes away, leaving half of his fortune to his eldest son, third to his middle son, and ninth to his smallest.

Upon opening the stable, they realize that the old man had 17 camels. Image
This is a problem, as they cannot split 17 camels into 1/2, 1/3, and 1/9 without cutting some in half.

So, they turn to the wise neighbor for advice. Image
Read 18 tweets
Oct 9
Matrix multiplication is not easy to understand.

Even looking at the definition used to make me sweat, let alone trying to comprehend the pattern. Yet, there is a stunningly simple explanation behind it.

Let's pull back the curtain! Image
First, the raw definition.

This is how the product of A and B is given. Not the easiest (or most pleasant) to look at.

We are going to unwrap this. Image
Here is a quick visualization before the technical details.

The element in the i-th row and j-th column of AB is the dot product of A's i-th row and B's j-th column. Image
Read 16 tweets
Oct 8
Graph theory will seriously enhance your engineering skills.

Here's why you must be familiar with graphs: Image
What do the internet, your brain, the entire list of people you’ve ever met, and the city you live in have in common?

These are all radically different concepts, but they share a common trait.

They are all networks that establish relationships between objects. Image
As distinct as these things seem to be, they share common properties.

For example, the meaning of “distance” is different for

• Social networks
• Physical networks
• Information networks

But in all cases, there is a sense in which some objects are “close” or “far”. Image
Read 14 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(