Do you like Star Wars?
Let's talk about John, a machine learning engineer.
He is building a model to predict whether people will like a future Star Wars episode.
This is the story of how John screwed things up.
☕️🧵👇
Let's talk about John, a machine learning engineer.
He is building a model to predict whether people will like a future Star Wars episode.
This is the story of how John screwed things up.
☕️🧵👇
They are screening the new Star War episode, and the theater seems like a good place to collect some information to create the model.
John goes right in and gives an optional survey to the people sitting in the middle of the theater right before the movie starts.
John goes right in and gives an optional survey to the people sitting in the middle of the theater right before the movie starts.
John does the same for a couple of weeks, then goes back to his computer and puts together a model.
It turns out that the model sucks.
The new episode doesn't do well despite the model predicting the opposite.
What happened?
It turns out that the model sucks.
The new episode doesn't do well despite the model predicting the opposite.
What happened?
John only sampled from people who decided to see the movie in the first place.
These results may not properly represent the views of the people who weren't interested in the movie and didn't go to the theater.
This is called "Coverage Bias."
These results may not properly represent the views of the people who weren't interested in the movie and didn't go to the theater.
This is called "Coverage Bias."
John gave the survey to the people sitting in the middle of the theater.
These are usually the most sought-after tickets. These people are probably the biggest Star Wars fans!
This is called "Sampling Bias."
These are usually the most sought-after tickets. These people are probably the biggest Star Wars fans!
This is called "Sampling Bias."
John used an optional survey to get the results, but only people with strong opinions are likely to participate.
Those who don't care enough will not offer their opinion, so the results may not properly represent everyone's point of view.
This is called "Participation Bias."
Those who don't care enough will not offer their opinion, so the results may not properly represent everyone's point of view.
This is called "Participation Bias."
These 3 are a form of Selection Bias, and it's essential we keep them in mind when working with data.
From politics to now-infamous public programs, we've all heard about the failures when the parties involved fail to take this seriously.
From politics to now-infamous public programs, we've all heard about the failures when the parties involved fail to take this seriously.
This is how John butchered an excellent initiative.
You don't want to be John, and you don't have to, because I'm here to help!
Follow me, and let's do this thing together: my specialty is to make machine learning suck a little bit less and share the journey with you.
🦕
You don't want to be John, and you don't have to, because I'm here to help!
Follow me, and let's do this thing together: my specialty is to make machine learning suck a little bit less and share the journey with you.
🦕
• • •
Missing some Tweet in this thread? You can try to
force a refresh
