Santiago Profile picture
Aug 12 18 tweets 6 min read Read on X
The single most undervalued fact of linear algebra:

Matrices are graphs, and graphs are matrices.

Encoding matrices as graphs is a cheat code, making complex behavior simple to study.

Let me show you how! Image
By the way, this thread is courtesy of @TivadarDanka. He allowed me to republish it.

3 years ago, he started writing a book about the mathematics of Machine Learning.

It's the best book you'll ever read:



Nobody explains complex ideas like he does.tivadardanka.com/books/mathemat…
If you look at this example, you probably figured out the rule.

Each row is a node, and each element represents a directed and weighted edge. We omit any edges of zero elements.

The element in the 𝑖-th row and 𝑗-th column corresponds to an edge going from 𝑖 to 𝑗. Image
To unwrap the definition a bit, let's check the first row, which corresponds to the edges outgoing from the first node.

(Notice how there's no edge for the value 0.) Image
Similarly, the first column corresponds to the edges incoming to the first node. Image
Here is the full picture, with the nodes explicitly labeled. Image
Why is the directed graph representation beneficial?

For example, the powers of the matrix correspond to walks in the graph.

Take a look at the elements of the square matrix. All possible 2-step walks are accounted for in the sum defining the elements of A². Image
If the directed graph represents the states of a Markov chain, the square of its transition probability matrix essentially shows the probability of the chain having some state after two steps.
There is much more to this connection.

For instance, it gives us a deep insight into the structure of nonnegative matrices.

To see what graphs show about matrices, let's talk about the concept of strongly connected components.
A directed graph is strongly connected if every node can be reached from every other node.

If this is not true, the graph is not strongly connected.

Below, you can see an example of both. Image
Matrices that correspond to strongly connected graphs are called irreducible. All other nonnegative matrices are called reducible.

Soon, we'll see why.

(For simplicity, I assumed each edge to have a unit weight, but each weight can be an arbitrary nonnegative number.) Image
Back to the general case!

Even though not all directed graphs are strongly connected, we can partition the nodes into strongly connected components. Image
Let's label the nodes of this graph and construct the corresponding matrix!

(For simplicity, assume that all edges have unit weight.)

Do you notice a pattern? Image
The corresponding matrix of our graph can be reduced to a simpler form!

Its diagonal comprises blocks whose graphs are strongly connected. (That is, the blocks are irreducible.) Furthermore, the block below the diagonal is zero. Image
In general, this block-matrix structure is called the Frobenius normal form. Image
Let's reverse the question: can we transform an arbitrary nonnegative matrix into the Frobenius normal form?

Yes, and with the help of directed graphs, this is much easier to show than purely using algebra. Image
This is just the tip of the iceberg. For example, with the help of matrices, we can define the eigenvalues of graphs!

Utilizing the relation between matrices and graphs has been extremely profitable for both graph theory and linear algebra.
This thread is just ~30% of the full post, which you can find on Tivadar's book.

You won't find better explanations anywhere else:



Trust me on this one. This is the book you want to read.tivadardanka.com/books/mathemat…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Santiago

Santiago Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @svpino

Jul 12
A common fallacy:

If it's raining, the sidewalk is wet. But if the sidewalk is wet, is it raining?

Reversing the implication is called "affirming the consequent." We usually fall for this.

But surprisingly, it's not entirely wrong!

Let's explain it using Bayes Theorem:

1/10 Image
This explanation is courtesy of @TivadarDanka. He allowed me to republish it.

He is writing a book about the mathematics of Machine Learning. It's the best book I've read:



Nobody explains complex ideas like he does.

2/10tivadardanka.com/books/mathemat…
We call propositions of the form "if A, then B" implications.

We write them as "A → B," and they form the bulk of our scientific knowledge.

For example:

"If X is a closed system, then the entropy of X cannot decrease" is the second law of thermodynamics.

3/10
Read 10 tweets
Jun 12
Some of the skills you need to start building AI applications:

• Python and SQL
• Transformer and diffusion models
• LLMs and fine-tuning
• Retrieval Augmented Generation
• Vector databases

Here is one of the most comprehensive programs that you'll find online:
"Generative AI for Software Developers" is a 4-month online course.

It's a 5 to 10-hour weekly commitment, but you can dedicate as much time as you want to finish early.

Here is the link to the program:

I also have a PDF with the syllabus:bit.ly/4aNOJdy


I'm a huge fan of online education, but most of it is all over the place and mostly theoretical.

This program is different:

You'll work on 4 different hands-on projects. You'll learn practical skills you can use at the office right away.cdn.sanity.io/files/tlr8oxjg…
Read 6 tweets
Jun 10
There's a stunning, simple explanation behind matrix multiplication.

This is the first time this clicked on my brain, and it will be the best thing you read all week.

Here is a breakdown of the most crucial idea behind modern machine learning:

1/15 Image
This explanation is courtesy of @TivadarDanka. He allowed me to republish it

3 years ago, he started writing a book about the mathematics of Machine Learning.

It's the best book you'll ever read:



Nobody explains complex ideas like he does.

2/15tivadardanka.com/books/mathemat…
Let's start with the raw definition of the product of A and B.

This looks horrible and complicated.

Let's unwrap it step by step.

3/15 Image
Read 15 tweets
May 28
This assistant has 169 lines of code:

• Gemini Flash
• OpenAI Whisper
• OpenAI TTS API
• OpenCV

GPT-4o is slower than Flash, more expensive, chatty, and very stubborn (it doesn't like to stick to my prompts).

Next week, I'll post a step-by-step video on how to build this.
The first request takes longer (warming up), but things work faster from that point.

Few opportunities to improve this:

1. Stream answers from the model (instead of waiting for the full answer.)

2. Add the ability to interrupt the assistant.

3. Whisper running on GPU
Unfortunately, no local modal supports text+images (as far as I know,) so I'm stuck running online models.

The TTS API (synthesizing text to audio) can also be replaced by a local version. I tried, but the available voices suck (too robotic), so I kept OpenAI's.
Read 4 tweets
May 25
I’m so sorry about anyone who bought the rabbit r1.

It’s not just that the product is non-functional (as we learned from all the reviews), the real problem is that the whole thing seems to be a lie.

None of what they pitched exists or functions the way they said. Image
They sold the world on a Large Action Model (LAM), an intelligent AI model that would understand applications and execute the actions requested by the user.

In reality, they are using Playwright, a web automation tool.

No AI. Just dumb, click-around, hard-coded scripts. Image
Their foundational AI model is just ChatGPT + scripts.

Rabbit’s founder lied on their marketing videos, during interviews, when he presented the product, and lied on Discord when answering questions from early supporters.

And that’s just the beginning:
Read 4 tweets
Mar 31
What a week, huh?

1. Mojo 🔥 went open-source
2. Claude 3 beats GPT-4
3. $100B supercomputer from MSFT and OpenAI
4. Andrew Ng and Harrison Chase discussed AI Agents
5. Karpathy talked about the future of AI
...

And more.

Here is everything that will keep you up at night:
Mojo 🔥, the programming language that turns Python into a beast, went open-source.

This is a huge step and great news for the Python and AI communities!

With Mojo 🔥 you can write Python code or scale all the way down to metal code. It's fast!

modular.com/blog/the-next-…
Claude 3 is the best model in the market right now, overtaking GPT-4.

Claude 3 Opus is #1 in the Arena Leaderboard (beating GPT-4.)

Opus is a huge model, but Claude 3 Haiku is cheap and fast. And it's also beating GPT-4 0613!

Read 10 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(