Post

How to get URL link on X (Twitter) App

On the Twitter thread, click on or icon on the bottom
Click again on or Share Via icon
Click on Copy Link to Tweet
Paste it above and click "Unroll Thread"!
More info at Twitter Help

Tivadar Danka

@TivadarDanka

Sep 20, 2021 • 14 tweets • 4 min read • Read on X

Scrolly

What do you get when you let a monkey randomly smash the buttons on a typewriter?

Hamlet from Shakespeare, of course. And Romeo and Juliet. And every other finite string that is possible.

Don't believe me? Keep reading. ↓

Let's start at the very beginning!

Suppose that I have a coin that, when tossed, has a 1/2 probability of coming up heads and a 1/2 probability of coming up tails.

If I start tossing the coin and tracking the result, what is the probability of 𝑛𝑒𝑣𝑒𝑟 having heads?

To answer this, first, we calculate the probability of no heads in 𝑛 tosses. (That is, the probability of 𝑛 tails.)

Since tosses are independent of each other, we can just multiply the probabilities for each toss together.

By letting 𝑛 to infinity, we obtain that the probability of never tossing heads is zero.

That is, we are going to have heads come up eventually with probability 1.

Instead of coin tosses, we can talk about arbitrary events.

If an event has a nonzero probability and you have infinite attempts, 𝑖𝑡 𝑤𝑖𝑙𝑙 ℎ𝑎𝑝𝑝𝑒𝑛 with probability 1.

Now, let's apply that to our monkey, infinitely typing away at the typewriter.

What is the probability that six random consecutive keystrokes result in the string "Hamlet"?

First, each keystroke matching the right character is 1/(number of keys).

Because the keystrokes are independent, the probability of a given string is the product of the probabilities for each keystroke matching the individual character.

Now, let's calculate the probability that the entire Hamlet play by Shakespeare is typed randomly.

Since the entire play has 194270 characters, and there are 100 possible keys to hit, this probability is extremely small.

Still, it is larger than zero.

Thus, if our monkey keeps typing infinitely, the entire Hamlet play will appear somewhere. (Along with every other finite string you can imagine.)

However, this takes a 𝑣𝑒𝑟𝑦 long time on average.

If the probability of a given string occurring is 𝑝, the expected number of attempts to randomly generate it is 1/𝑝.

So, if 𝑝 is as small as randomly typing the entire Hamlet play, then 1/𝑝 is going to be astronomical.

https://twitter.com/TivadarDanka/status/1359876189943382017

(If you are not familiar with the concept of expected values, take a look at the simple explanation I posted a while ago.)

https://twitter.com/TivadarDanka/status/1359876189943382017

It states that given infinite time, a monkey randomly smashing the keys of a typewriter will type any given text.

Next time when you say, "even a monkey can do it", be careful. Monkeys can do a lot.

Recently, I have been thinking about probability a lot.

In fact, I am writing the probability theory chapters of my book, Mathematics of Machine Learning. The early access is just out, where I publish one chapter every week.

tivadar.gumroad.com/l/mathematics-…

I post several threads like this every week, diving deep into concepts in machine learning and mathematics.

If you have enjoyed this, make sure to follow me and stay tuned for more!

The theory behind machine learning is beautiful, and I want to show this to you.

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @TivadarDanka

Tivadar Danka

@TivadarDanka

Dec 4, 2023

Understanding graph theory will seriously enhance your engineering skills; you must absolutely be familiar with them.

Here's a graph theory quickstart, in collaboration with Alejandro Piad Morffis.

Read on:

What do the internet, your brain, the entire list of people you’ve ever met, and the city you live in have in common?

These are all radically different concepts, but they share a common trait.

They are all networks that establish relationships between objects.

As distinct as these things seem to be, they share common properties.

For example, the meaning of “distance” is different for

• physical networks,
• information netorks,
• orf social networks,

but in all cases, there is a sense in which some objects are “close” or “far”.

Read 15 tweets

Tivadar Danka

@TivadarDanka

Sep 13, 2023

Neural networks are stunningly powerful.

This is old news: deep learning is state-of-the-art in many fields, like computer vision and natural language processing. (But not everywhere.)

Why are neural networks so effective? I'll explain.

First, let's formulate the classical supervised learning task!

Suppose that we have a dataset D, where xₖ is a data point and yₖ is the ground truth.

The task is simply to find a function g(x) for which

• g(xₖ) is approximately yₖ,
• and g(x) is computationally feasible.

To achieve this, we fix a parametrized family of functions. For instance, linear regression uses this function family:

Read 19 tweets

Tivadar Danka

@TivadarDanka

Sep 12, 2023

A question we never ask:

"How large that number in the Law of Large Numbers is?"

Sometimes, a thousand samples are large enough. Sometimes, even ten million samples fall short.

How do we know? I'll explain.

First things first: the law of large numbers (LLN).

Roughly speaking, it states that the averages of independent, identically distributed samples converge to the expected value, given that the number of samples grows to infinity.

We are going to dig deeper.

There are two kinds of LLN-s: weak and strong.

The weak law makes a probabilistic statement about the sample averages: it implies that the probability of "the sample average falling farther from the expected value than ε" goes to zero for any ε.

Let's unpack this.

Read 15 tweets

Tivadar Danka

@TivadarDanka

Aug 24, 2023

With the power of mathematical induction, I'll prove that everyone has the same eye color.

Don't believe me? Read on.

(And see if you can spot the sleight of hand.)

To formalize the problem, define the proposition Aₙ by

Aₙ = "in a set of n people, everyone has the same eye color".

If n equals the human population of planet Earth, we get the original statement. We'll prove that Aₙ is true via induction.

Proof by induction works like climbing an infinite staircase.

First, we'll show A₁. Then, we'll show that if Aₙ is true, then Aₙ₊₁ is true as well.

This way, Aₙ is true for any positive integer via the chain of implications

A₁ → A₂ → ... → Aₙ.

Read 13 tweets

Tivadar Danka

@TivadarDanka

Aug 21, 2023

The single biggest argument about statistics: is probability frequentist or Bayesian? It's neither, and I'll explain why.

Buckle up. Deep-dive explanation incoming.

First, let's look at what is probability.

Probability quantitatively measures the likelihood of events, like rolling six with a dice. It's a number between zero and one. This is independent of interpretation; it’s a rule set in stone.

In the language of probability theory, the events are formalized by sets within an event space.

(The event space is also a set, usually denoted by Ω.)

Read 34 tweets

Tivadar Danka

@TivadarDanka

Aug 8, 2023

The Japanese multiplication method makes everybody feel "I wish they taught math like this in school."

It's not just a cute visual tool: it illuminates how and why long multiplication works.

Here is the full story.

First, the Japanese multiplication method.

The first operand (21 in our case) is represented by two groups of lines: two lines in the first (1st digit), and one in the second (2nd digit).

One group for each digit.