MORE PARAMETERS DON’T AWAYS LEAD TO MORE ACCURACY
(thread)
Italy assigned to each region a COVID risk level based on a complex model with 21 parameters.
Many experts claimed that the more parameters, the more precise the model.
Is it true, though?
(1/6)
(thread)
Italy assigned to each region a COVID risk level based on a complex model with 21 parameters.
Many experts claimed that the more parameters, the more precise the model.
Is it true, though?
(1/6)
2/ In most real-world cases, more parameters lead to worse predictions.
See the example below (from my gum.co/heuristic)
See the example below (from my gum.co/heuristic)
3/ If we order the 21 parameters by descending predictive power, and apply it to the data used to build the model, we discover that:
- The first parameter alone predicts the outcome of many regions.
- The last parameter, at most, helps predicting the outcome of a single region.
- The first parameter alone predicts the outcome of many regions.
- The last parameter, at most, helps predicting the outcome of a single region.
4/ With an informal reversal, we could say that the first parameter is validated using many data points, whereas the last parameter is validated using a single data point - which might be noisy!
(For detailed explanation, see the screenshots above.)
(For detailed explanation, see the screenshots above.)
5/ Hence, no, it’s not a given that more parameters in a predictive model increase its accuracy.
Often, they decrease it.
Often, they decrease it.
6/ Humans and animals use heuristics and rules of thumb not (just) because they are fast, but (also) because they have very few parameters and are thus very resistant to noise.
Whereas complex models are prone to overfitting to data which is not representative of the future.
Whereas complex models are prone to overfitting to data which is not representative of the future.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
