May 27 16 tweets 32 min read
2/16

"roc_auc_score" is defined as the area under the ROC curve, which is the curve having False Positive Rate on the x-axis and True Positive Rate on the y-axis at all classification thresholds.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python
4/16

AUC is equivalent to the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Math #Data #DataAnalytics
5/16

A model whose predictions are 100% wrong has an AUC of 0. and one whose predictions are 100% correct has an AUC of 1

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Math #Data #DataAnalytics #pythoncode #AI
6/16

In other words - roc_auc_score coincides with “the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one”.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #DataScientist #Statistics
7/16

AUC is scale-invariant. It measures how well predictions are ranked, rather than their absolute values.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Math #Data #DataAnalytics #pythoncode #AI
8/16

AUC is classification-threshold-invariant. It measures the quality of the model's predictions irrespective of what classification threshold is chosen.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Math #Data
10/16

Scale invariance is not always desirable. e.g, sometimes we really do need well calibrated probability outputs, and AUC won’t tell us about that.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Math #Data
11/16

Classification-threshold invariance is not always desirable, in cases where there are wide disparities in the cost of false negatives vs. false positives, it may be critical to minimize one type of classification error.

#DataScience #MachineLearning #100DaysOfMLCode
12/16

e.g, in email spam detection, you likely want to prioritize minimizing false positives (i.e. an Email is NOT spam, but its positively determined as a spam and hence moved to span folder).

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Statistics
13/16

Even if that results in a significant increase of false negatives. (An email is indeed spam, but model determines it to be negative, i.e. Not-Spam). AUC isn't a useful metric for this type of optimization.

#DataScience #MachineLearning #Statistics #DeepLearning
14/16

How to use the AUC ROC curve for the multi-class model ?

In a multi-class model, we can plot the N number of AUC ROC Curves for N number classes using the One vs Rest methodology.

#DataScience #MachineLearning #100DaysOfMLCode #Python #DataScientist #Statistics
15/16

“One vs Rest” is a method to evaluate multiclass models by comparing each class against all the others at the same time. Here we take one class and consider it as our “positive” class, while all the others (the rest) are considered as the “negative” class.

#DataScience
16/16

e.s. if you have three classes named X, Y, and Z, you will have one ROC for X classified against Y and Z, another ROC for Y classified against X and Z, and the third one of Z classified against Y and X.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

# More from @rohanpaul_ai

May 28
1/ "Software is eating the world. Machine learning is eating software. Transformers are eating machine learning."

Let's understand what these Transformers are all about

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataAnalytics
2/ #Transformers architecture follows Encoder and Decoder structure.

The encoder receives input sequence and creates intermediate representation by applying embedding and attention mechanism.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI
3/ Then, this intermediate representation or hidden state will pass through the decoder, and the decoder starts generating an output sequence.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics
May 28
But what p-value means in #MachineLearning - A thread

It tells you how likely it is that your data could have occurred under the null hypothesis

1/n

#DataScience #DeepLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data #Math #Stat
2/n
What Is a Null Hypothesis?

A null hypothesis is a type of statistical hypothesis that proposes that no statistical significance exists in a set of given observations.

#DataScience #MachineLearning #100DaysOfMLCode #Python #stat #Statistics #Data #AI #Math #deeplearning
3/n
A P-value is the probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis

#DataScience #MachineLearning #100DaysOfMLCode #Python #DataScientist #Statistics #Data #DataAnalytics #AI #Math
May 28
1/ One way to test whether a time series is stationary is to perform an augmented Dickey-Fuller test - A Thread

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics #programming #ArtificialIntelligence
2/ H0: The time series is non-stationary. In other words, it has some time-dependent structure and does not have constant variance over time.

HA: The time series is stationary.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist
3/ If the p-value from the test is less than some significance level (e.g. α = .05), then we can reject the null hypothesis and conclude that the time series is stationary.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist
May 28
Kullback-Leibler (KL) Divergence - A Thread

It is a measure of how one probability distribution diverges from another expected probability distribution.

#DataScience #Statistics #DeepLearning #ComputerVision #100DaysOfMLCode #Python #programming #ArtificialIntelligence #Data
May 27
2/ It is important to standardize variables before running Cluster Analysis. It is because cluster analysis techniques depend on the concept of measuring the distance between the different observations we're trying to cluster.

#DataScience #MachineLearning #DeepLearning
3/ If a variable is measured at a higher scale than the other variables, then whatever measure we use will be overly influenced by that variable.

#DataScience #MachineLearning #DeepLearning #100DaysOfMLCode #Python #pythoncode #AI #DataScientist #DataAnalytics #Statistics
May 27
Did you know how TensorFlow can run on a single mobile device as well as on an entire data center? Read this thread

1/n

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data
2/n
Google has designed TensorFlow such that it is capable of dividing a large model graph whenever needed.

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data #Math #Stat #AI
3/n
It assigns special SEND and RECV nodes whenever a graph is divided between multiple devices (CPUs or GPUs).

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data #Math #Stat #AI