The concept that helped me go from bad models to good models: Bias and Variance.
In 4 minutes, I'll share 4 years of experience in managing bias and variance in my machine learning models. Let's go. 🧵
1. Generalization:
Bias and variance control your models ability to generalize on new, unseen data, not just the data it was trained on. The goal in machine learning is to build models that generalize well. To do so, I manage bias and variance.
2. Low vs High Bias:
Models with low bias are usually complex and can capture the underlying patterns in data very well. They are flexible enough to fit the training data closely. Models with high bias are overly simple and cannot capture the complexity in the data. They often underfit the training data, meaning they perform poorly even on the data they were trained on.
K-means is an essential algorithm for Data Science.
But it's confusing for beginners.
Let me demolish your confusion:
1. K-Means
K-means is a popular unsupervised machine learning algorithm used for clustering. It's a core algorithm used for customer segmentation, inventory categorization, market segmentation, and even anomaly detection.
2. Unsupervised:
K-means is an unsupervised algorithm used on data with no labels or predefined outcomes. The goal is not to predict a target output, but to explore the structure of the data by identifying patterns, clusters, or relationships within the dataset.
Correlation is the skill that has singlehandedly benefitted me the most in my career.
In 3 minutes I'll demolish your confusion (and share strengths and weaknesses you might be missing).
Let's go:
1. Correlation:
Correlation is a statistical measure that describes the extent to which two variables change together. It can indicate whether and how strongly pairs of variables are related.
2. Types of correlation:
Several types of correlation are used in statistics to measure the strength and direction of the relationship between variables. The three most common types are Pearson, Spearman Rank, and Kendall's Tau. We'll focus on Pearson since that is what I use 95% of the time.