Thread by @v_vashishta

What is the difference between a predictive problem and a causal inference problem? This is an essential differentiation for data scientists, and even very smart people botch the answer. 2 very smart authors did just that. Let me explain.
1/13
#DataScience #MachineLearning

They proposed 2 questions:

1. Should I hire more college graduates?
2. Should I subsidize college degrees for my employees?
2/13
#DataScience #MachineLearning

In the article, they said question 1 is a prediction problem and question 2 is a causal inference problem. They are both causal inference problems because they ask the data scientist to prescribe a policy.
3/13
#DataScience #MachineLearning

By answering yes or no to either question, I tell the decision maker that I have evidentiary support for the decision.
4/13
#DataScience #MachineLearning

Rephrase question 1 to “Are college graduates better (as defined by some set of metrics) employees?” and the question is now a predictive one. As a data scientist, I can use correlation to support my assertions. What’s the difference?
5/13
#DataScience #MachineLearning

In the rephrased question, I must support a relationship between variables. In the original questions, I must support a relationship between a decision and an outcome. Those have two distinctly different reliability requirements.
6/13
#DataScience #MachineLearning

Why? The relationship between variables implies my model accurately describes the data, and I have a confidence level that the description will hold in the future.
7/13
#DataScience #MachineLearning

The relationship between decision and outcome implies that my model is as accurate or more accurate than a person’s heuristics.
8/13
#DataScience #MachineLearning

The relationship between variables is me proposing a set of KPIs or metrics and claiming, “If you make your decision based on these, your decision quality will be high.”
9/13
#DataScience #MachineLearning

The relationship between decision and outcome is me claiming, “This decision is high quality.” To support that assertion, I must be confident that the model makes a better decision than a person using the best available KPIs.
10/13
#DataScience #MachineLearning

That’s rarely true with simple correlation unless the number of variables involved in the decision is so high that people given that data make worse decisions than people who are not.
11/13
#DataScience #MachineLearning

Otherwise, I need to establish causal relationships to support the model's understanding of the system being better than a decision maker's.
12/13
#DataScience #MachineLearning

My substack newsletter is filled with deep dives into advanced topics. Take a look and subscribe here: vinvashishta.substack.com
13/13
#DataScience #MachineLearning