Thread by @ReformedTrader

1/ Robust Beauty of Improper Linear Models in Decision-Making (Dawes)

"Even improper linear models are superior to clinical intuition when predicting a numerical criterion from numerical predictors. In fact, unit (i.e., equal) weighting is quite robust."

uwe-mortensen.de/DawesRobustBea…

2/ "In proper linear models, predictor variables are weighted such that the resulting linear composite optimally predicts some criterion of interest; examples of proper linear models are standard regression analysis, discriminant function analysis, and ridge regression analysis.

3/ "Research summarized in Paul Meehl's book on clinical vs. statistical prediction—and a plethora of research—indicate that when a numerical criterion (e.g., graduate GPA) is to be predicted from numerical predictor variables, proper linear models outperform clinical intuition.

4/ "Improper linear models are those in which the weights of the predictor variables are obtained by some nonoptimal method; for example, they may be obtained on the basis of intuition, derived from simulating a clinical judge's predictions, or set to be equal."

5/ "People are important. The model may integrate the information in an optimal manner, but it is always the individual (judge, clinician) who chooses variables. Moreover, the human judge knows the directional relationship between the predictors and the criterion of interest."

6/ "In multiple regression, b weights are notoriously unstable; the ratio of observations to predictors should be as high as 15 or 20 to 1 before b weights, which are the optimal weights, do better on cross validation than do simple unit weights."

7/ "Bootstrapping models use weights derived from judges; because they are not derived from the relationship between the predictor and criterion variables themselves, the resulting models are improper. Yet they consistently do better than the judges from which they were derived."

8/ "Random linear models perform about as well as the paramorphic models of the judges; these averages are presented in the third column of the table. Equal-weighting models [after standardizing the variables], presented in the fourth column, do even better."

9/ "Linear models are robust over deviations from optimal weighting. In other words, the bootstrapping finding, at least in these studies, has simply been a reaffirmation of the earlier finding that proper linear models are superior to human judgments."

10/ "People who argue that different criteria/judges/variables/time frames would produce different results have had 25 years in which to produce examples and have failed to do so.

"The illusion of good judgment may be reinforced when predictions have self-fulfilling effects."

11/ "Statistical prediction, because it includes the specification of exactly how poorly we can predict, bluntly strikes us with the fact that life is not all that predictable. Unsystematic clinical prediction, in contrast, allows us to hold to a comforting illusion."

12/ "The GRE is a carefully devised test. Do we really believe we can do a better job by a 10-minute folder evaluation or half-hour interview? Such cognitive conceit is unethical, especially given the complete lack of evidence that we do better than the linear equation does."

13/ Related reading:

Anomalies and News
"Analysts are normally wrong stocks' expected returns. Over pessimism/over-optimism could contribute to anomaly returns."


Superforecasting


Expert Political Judgment