Expected value is one of the most fundamental concepts in probability theory and machine learning.
Have you ever wondered what it really means and where does it come from?
The formula doesn't tell the entire story right away.
💡 Let's unravel what is behind the scenes! 💡
First, let's take a look at a simple example.
Suppose that we are playing a game. You toss a coin, and
• if it comes up heads, you win $1,
• but if it is tails, you lose $2.
Should you even play this game with me? 🤔
We are about to find out!
After 𝑛 rounds, your earnings can be calculated by the number of heads times 1 minus the number of tails times 2.
If we divide total earnings by 𝑛, we obtain the average earnings per round.
What happens if 𝑛 approaches infinity? 🤔
As you have probably guessed, the number of heads divided by the number of tosses will converge to the 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 of a single toss being heads.
In our case, this is 1/2.
(Similarly, tails/tosses also converges to 1/2.)
So, your average earnings per round are -1/2. This is the 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑣𝑎𝑙𝑢𝑒.
By the way, you definitely shouldn't play this game. 😉
💡 How can we calculate the expected value for a general case? 💡
Suppose that similarly to the previous example, the outcome of your experiments can be quantified. (Like throwing a dice or making a bet at the poker table.)
The expected value is just the average outcome you have per experiment when you let it run infinitely. ♾️🤯
The formula above is simply the expected value in English.
If we formally denote the variable describing the outcome of the experiment with 𝑋 and its possible values with 𝑥ᵢ, we get back the formula in the first tweet.
It looks much easier now, isn't it?
This concept came up recently when I gave this explanation to a friend.
Inspired by @haltakov and his awesome recent thread (
If you are building a product, chances are you severely underestimate the importance of idea validation. (Especially if you are a developer.)
Key business assumptions can flop because you fail to look at different angles.
What are some basic questions to ask?
🧵 A thread. 🧵
𝐀𝐦 𝐈 𝐬𝐨𝐥𝐯𝐢𝐧𝐠 𝐚𝐧 𝐞𝐱𝐢𝐬𝐭𝐢𝐧𝐠 𝐩𝐫𝐨𝐛𝐥𝐞𝐦?
Often, the problem is not important enough to justify the existence of a solution. This is the most basic trap to fall for: there is no market need for the product.
Take a look at the top 20 reasons why startups fail by @CBinsights. The number 1 is no market need, causing around 43% percent of failures.
Have you ever implemented a dynamic function dispatcher in Python, where you can register functions at runtime using the decorator syntax? (Like the routers for FastAPI.)
I did this recently, and I am going to teach you how to do it! I'll walk you through it in the thread below.
We will build an event handler to catch arbitrary events and dynamically execute a function to handle the event.
(A good example is catching events in a webhook listener.)
Time to use some decorator magic!
The usage is straightforward: 1) instantiate the EventHandler, 2) register handler functions for specific events, 3) pass the event to the EventHandler instance when caught.
Neural networks are getting HUGE. In their @stateofaireport 2020, @NathanBenaich and @soundboy visualized how the number of parameters grew for breakthrough architectures. The result below is staggering.
What can you do to compress neural networks?
👇A thread.
1⃣ Neural network pruning: iteratively removing connections after training. Turns out that in some cases, 90%+ of the weights can be removed without noticeable performance loss.
A few selected milestone papers:
📰Optimal Brain Damage by @ylecun, John S. Denker, and @SaraASolla. As far as I know, this was the one where the idea was introduced.