"Remember, these models are always wrong.” -@drsanjaygupta
This kind of statement reinforces our misconception of what models are: *decision tools* that give our *best estimates* of future given certain assumptions/inputs— helping us improve decision making under uncertainty.
This kind of statement reinforces our misconception of what models are: *decision tools* that give our *best estimates* of future given certain assumptions/inputs— helping us improve decision making under uncertainty.
THREAD: 👇
These models have lots of inputs, which change based on new information like states opening back up or asymptomatic transmission etc.
2/10
These models have lots of inputs, which change based on new information like states opening back up or asymptomatic transmission etc.
2/10
Updating a model doesn’t make previous projections wrong, since they were forecasts based on current information and assumptions and *SHOULD change* as new information reveals itself.
3/9
3/9
In fact, even the IMHE model that Gupta is talking about offers an upper and lower bound, yet media only reports the point forecast. The range reflects the uncertainty, which doesn’t get communicated well to the public.
4/9
4/9
Can a model be a bad model? Sure. The assumption can turn out to be wrong. The weighting of the variables can be poor etc. But updating a model wrong. In fact NOT updating a model based on new information (Georgia just opened up!) is what would make it bad.
5/9
5/9
That would be like saying if there is a forecast of a 30% chance of rain, and it doesn’t rain, the model is wrong.
Or if it forecast a 30% chance of rain a week out and updated two days later to a 50% chance of rain, that the model would be wrong.
6/9
Or if it forecast a 30% chance of rain a week out and updated two days later to a 50% chance of rain, that the model would be wrong.
6/9
You could only find out if a weather model is a bad model if over a large enough sample size of days where it forecast a 30% chance of rain, it didn’t rain 30% of the time.
7/9
7/9
If we were making decisions under conditions of certainty, we would need a model. We would know the answer already.
Under uncertainty, we need to make educated guesses to guide our decision making, always updated as we learn more.
8/9
Under uncertainty, we need to make educated guesses to guide our decision making, always updated as we learn more.
8/9
That’s the whole point of a model.
Good ones update.
👆 @drsanjaygupta @PTetlock
#Modeling #DecisionTools #Forecasting #DecisionMaking #Covid19 #Coronavirus
End thread. [9/9]
Good ones update.
👆 @drsanjaygupta @PTetlock
#Modeling #DecisionTools #Forecasting #DecisionMaking #Covid19 #Coronavirus
End thread. [9/9]
