Discover and read the best of Twitter Threads about #RepostFriday
Most recents (7)
Dealing with imbalanced datasets π βοΈ π
Real world datasets are often imbalanced - some of the classes appear much more often than others.
The problem? You ML model will likely learn to only predict the dominant classes.
What can you do about it? π€
Thread 𧡠#RepostFriday
Real world datasets are often imbalanced - some of the classes appear much more often than others.
The problem? You ML model will likely learn to only predict the dominant classes.
What can you do about it? π€
Thread 𧡠#RepostFriday
Example π¦
We will be dealing with an ML model to detect traffic lights for a self-driving car π€π
Traffic lights are small so you will have much more parts of the image that are not traffic lights.
Furthermore, yellow lights π‘ are much rarer than green π’ or red π΄.
We will be dealing with an ML model to detect traffic lights for a self-driving car π€π
Traffic lights are small so you will have much more parts of the image that are not traffic lights.
Furthermore, yellow lights π‘ are much rarer than green π’ or red π΄.
The problem β‘
Imagine we train a model to classify the color of the traffic light. A typical distribution will be:
π΄ - 56%
π‘ - 3%
π’ - 41%
So, your model can get to 97% accuracy just by learning to distinguish red from green.
How can we deal with this?
Imagine we train a model to classify the color of the traffic light. A typical distribution will be:
π΄ - 56%
π‘ - 3%
π’ - 41%
So, your model can get to 97% accuracy just by learning to distinguish red from green.
How can we deal with this?
s this formula difficult? π€
This is the formula for Gradient Descent with Momentum as presented in Wikipedia.
It may look intimidating at first, but I promise you that by the end of this thread it will be easy to understand!
Thread π
#RepostFriday
This is the formula for Gradient Descent with Momentum as presented in Wikipedia.
It may look intimidating at first, but I promise you that by the end of this thread it will be easy to understand!
Thread π
#RepostFriday
The Basis β»οΈ
Let's break it down! The basis is this simple formula describing an iterative optimization method.
We have some weights (parameters) and we iteratively update them in some way to reach a goal
Iterative methods are used when we cannot compute the solution directly
Let's break it down! The basis is this simple formula describing an iterative optimization method.
We have some weights (parameters) and we iteratively update them in some way to reach a goal
Iterative methods are used when we cannot compute the solution directly
Gradient Decent Update π
We define a loss function describing how good our model is. We want to find the weights that minimize the loss (make the model better).
We compute the gradient of the loss and update the weights by a small amount (learning rate) against the gradient.
We define a loss function describing how good our model is. We want to find the weights that minimize the loss (make the model better).
We compute the gradient of the loss and update the weights by a small amount (learning rate) against the gradient.
Machine Learning in the Real World π§ π€
ML for real-world applications is much more than designing fancy networks and fine-tuning parameters.
In fact, you will spend most of your time curating a good dataset.
Let's go through the process together π
#RepostFriday
ML for real-world applications is much more than designing fancy networks and fine-tuning parameters.
In fact, you will spend most of your time curating a good dataset.
Let's go through the process together π
#RepostFriday
Collect Data π½
We need to represent the real world as accurately as possible. If some situations are underrepresented we are introducing Sampling Bias.
Sampling Bias is nasty because we'll have high test accuracy, but our model will perform badly when deployed.
π
We need to represent the real world as accurately as possible. If some situations are underrepresented we are introducing Sampling Bias.
Sampling Bias is nasty because we'll have high test accuracy, but our model will perform badly when deployed.
π
Traffic Lights π¦
Let's build a model to recognize traffic lights for a self-driving car. We need to collect data for different:
βͺοΈ Lighting conditions
βͺοΈ Weather conditions
βͺοΈ Distances and viewpoints
βͺοΈ Strange variants
And if we sample only π¦ we won't detect π₯ π€·ββοΈ
π
Let's build a model to recognize traffic lights for a self-driving car. We need to collect data for different:
βͺοΈ Lighting conditions
βͺοΈ Weather conditions
βͺοΈ Distances and viewpoints
βͺοΈ Strange variants
And if we sample only π¦ we won't detect π₯ π€·ββοΈ
π
Machine Learning Formulas Explained! π¨βπ«
This is the formula for the Binary Cross Entropy Loss. It is commonly used for binary classification problems.
It may look super confusing, but I promise you that it is actually quite simple!
Let's go step by step π
#RepostFriday
This is the formula for the Binary Cross Entropy Loss. It is commonly used for binary classification problems.
It may look super confusing, but I promise you that it is actually quite simple!
Let's go step by step π
#RepostFriday
The Cross-Entropy Loss function is one of the most used losses for classification problems. It tells us how well a machine learning model classifies a dataset compared to the ground truth labels.
The Binary Cross-Entropy Loss is a special case when we have only 2 classes.
π
The Binary Cross-Entropy Loss is a special case when we have only 2 classes.
π
The most important part to understand is this one - this is the core of the whole formula!
Here, Y denotes the ground-truth label, while ΕΆ is the predicted probability of the classifier.
Let's look at a simple example before we talk about the logarithm... π
Here, Y denotes the ground-truth label, while ΕΆ is the predicted probability of the classifier.
Let's look at a simple example before we talk about the logarithm... π
There are two problems with ROC curves
β They don't work for imbalanced datasets
β They don't work for object detection problems
So what do we do to evaluate our machine learning models properly in these cases?
We use a Precision-Recall curve.
Thread π
#RepostFriday
β They don't work for imbalanced datasets
β They don't work for object detection problems
So what do we do to evaluate our machine learning models properly in these cases?
We use a Precision-Recall curve.
Thread π
#RepostFriday
Last week I wrote another detailed thread on ROC curves. I recommend that you read it first if you don't know what they are.
Then go on π
Then go on π
β Problem 1 - Imbalanced Data
ROC curves measure the True Positive Rate (also known as Accuracy). So, if you have an imbalanced dataset, the ROC curve will not tell you if your classifier completely ignores the underrepresented class.
Let's take an example confusion matrix π
ROC curves measure the True Positive Rate (also known as Accuracy). So, if you have an imbalanced dataset, the ROC curve will not tell you if your classifier completely ignores the underrepresented class.
Let's take an example confusion matrix π
Did you ever want to learn how to read ROC curves? ππ€
This is something you will encounter a lot when analyzing the performance of machine learning models.
Let me help you understand them π
#RepostFriday
This is something you will encounter a lot when analyzing the performance of machine learning models.
Let me help you understand them π
#RepostFriday
What does ROC mean?
ROC stands for Receiver Operating Characteristic but just forget about it. This is a military term from the 1940s and doesn't make much sense today.
Think about these curves as True Positive Rate vs. False Positive Rate plots.
Now, let's dive in π
ROC stands for Receiver Operating Characteristic but just forget about it. This is a military term from the 1940s and doesn't make much sense today.
Think about these curves as True Positive Rate vs. False Positive Rate plots.
Now, let's dive in π
The ROC curve visualizes the trade-offs that a binary classifier makes between True Positives and False Positives.
This may sound too abstract for you so let's look at an example. After that, I encourage you to come back and read the previous sentence again!
Now the example π
This may sound too abstract for you so let's look at an example. After that, I encourage you to come back and read the previous sentence again!
Now the example π
How to evaluate your ML model? π
Your accuracy is 97%, so this is pretty good, right? Right? No! β
Just looking at the model accuracy is not enough. Let me tell you about some other metrics:
βͺοΈ Recall
βͺοΈ Precision
βͺοΈ F1 score
βͺοΈ Confusion matrix
Let's go π
#RepostFriday
Your accuracy is 97%, so this is pretty good, right? Right? No! β
Just looking at the model accuracy is not enough. Let me tell you about some other metrics:
βͺοΈ Recall
βͺοΈ Precision
βͺοΈ F1 score
βͺοΈ Confusion matrix
Let's go π
#RepostFriday
We'll use this example in the whole thread - classifying traffic light colors (e.g. for a self-driving car).
Yellow traffic lights appear much less often, so our dataset may look like this.
This means our model could reach 97% accuracy, by ignoring all π‘ lights. Not good!
π
Yellow traffic lights appear much less often, so our dataset may look like this.
This means our model could reach 97% accuracy, by ignoring all π‘ lights. Not good!
π
Let's assume now that we trained our model and we get the following predictions.
Do you think this model is good? How can we quantitatively evaluate its performance? How should it be improved?
Let's first discuss the possible error types π
Do you think this model is good? How can we quantitatively evaluate its performance? How should it be improved?
Let's first discuss the possible error types π
