Do you know what scares me? Data labeling in machine learning.
We don't talk enough about it, and yet we can't do anything unless we solve this first. Labeling enough data is expensive or even outright impossible.
Some ideas to solve this problem.
↓
We don't talk enough about it, and yet we can't do anything unless we solve this first. Labeling enough data is expensive or even outright impossible.
Some ideas to solve this problem.
↓
Let's start with an example:
You have terrain and weather information for different locations. Your goal is to build a model that predicts where to drill to find oil.
How do you label this data? You drill to find out where the oil is.
This is ridiculously expensive.
You have terrain and weather information for different locations. Your goal is to build a model that predicts where to drill to find oil.
How do you label this data? You drill to find out where the oil is.
This is ridiculously expensive.
To get around this problem, you need to minimize the number of labeled examples you need to build a good model.
1. Take the data
2. Select as few examples as possible
3. Drill those holes to come up with the labels
4. Train the model
How can you achieve #2?
1. Take the data
2. Select as few examples as possible
3. Drill those holes to come up with the labels
4. Train the model
How can you achieve #2?
Here's a possible solution:
Start by drilling a few holes to label some of the examples. Not too many, just enough to get a mediocre model started.
Use this model to label the rest of the dataset automatically.
The results of that model will be bad but useful.
Start by drilling a few holes to label some of the examples. Not too many, just enough to get a mediocre model started.
Use this model to label the rest of the dataset automatically.
The results of that model will be bad but useful.
Order the results of the model by their confidence score and take the top worst results.
For example:
• Sample A: 85% positive | 15% negative
• Sample B: 55% positive | 45% negative
We'd take Sample B.
Drill new holes on the locations represented by the samples you take.
For example:
• Sample A: 85% positive | 15% negative
• Sample B: 55% positive | 45% negative
We'd take Sample B.
Drill new holes on the locations represented by the samples you take.
You drill, you get new labels, your training data is now larger.
Retrain the model with the new dataset and repeat the whole process.
Stop when the model doesn't get any better.
Retrain the model with the new dataset and repeat the whole process.
Stop when the model doesn't get any better.
This is a really cool approach.
It's called "Active Learning," and I just described one of the ways you can implement it.
It helps you be really strategic about what data you label to train a model.
It's called "Active Learning," and I just described one of the ways you can implement it.
It helps you be really strategic about what data you label to train a model.
I go into more details about this approach in the latest issue of underfitted.io. Here is a link to it:
digest.underfitted.io/archive/594438
If you aren't subscribed yet, you are missing out.
It's free, and it puts one machine learning story right in your inbox every week.
digest.underfitted.io/archive/594438
If you aren't subscribed yet, you are missing out.
It's free, and it puts one machine learning story right in your inbox every week.
And if you enjoy a practical perspective on machine learning, follow me @svpino, and I'll make sure you get a constant stream of good content that will help us both get better at this thing.
Every week, one tweet at a time.
Every week, one tweet at a time.
Really good point here by Luke.
https://twitter.com/96stats/status/1390634724658348039
Love this perspective!
https://twitter.com/anabibikova/status/1390651678601449474
• • •
Missing some Tweet in this thread? You can try to
force a refresh
