Thread by @jposhaughnessy

1/ Human judgment is limited And We’ve Known That A Long Time--A Thread

In an earlier thread, I looked at an experiment that showed that most of us naturally find more complex explanations more compelling while eschewing the simpler methods that were actually more accurate.

2/ Now, let’s take a look at how we’ve known this for a long, long time.

In 1984, David Faust wrote in his book, “the Limits of Scientific Reasoning,” that: “human judgment is far more limited than we think. We have a surprisingly restricted capacity to manage or

3/ interpret complex information.“ Studying a wide range of professionals, from medical doctors making diagnoses to experts making predictions of job success in academic or military training, Faust found that human judges were consistently outperformed by simple actuarial models

4/ Like traditional money managers, most professionals cannot beat the passive implementation of time-tested formulas.

65 years ago, Another researcher, Paul Meehl, offered the first comprehensive review of statistical prediction (similar to an empirical, systematic approach)

5/ and clinical prediction (similar to an intuitive, traditional heuristic approach) in his 1954 study, “Clinical Versus Statistical Prediction: A Theoretical Analysis and Review of the Literature." He reviewed 20 studies that compared clinical and statistical predictions

6/for three things: academic success, response to electric shock therapy, and criminal recidivism. In almost every instance, Meehl found that simple actuarial models outperformed the human judges in predicting academic success in college, for example, a model using just high

7/ school grade point average and the level attained on an aptitude test (such as the SAT) outperformed the judgment of admissions officers at several colleges.

Robyn Dawes, in his book “House of Cards: Psychology and Psychotherapy Built on Myth,” tells us more.

8/ He refers to Jack Sawyer, a researcher who published a review of 45 studies comparing the two forecasting techniques: in none was the clinical, intuitive method – the one favored by most people – found to be superior. What’s more, Sawyer included instances in which the human

9/ judges had more information than the model and were given a result of the quantitative models before being asked for prediction. The actuarial models still beat the human judges!

I could go on and on with more studies and entire books outlining these findings, but you get

10/ the point--they *all* have similar findings and we've known about this for a long time. How is it that with all of this information, we keep ignoring the data and making decisions the same way? Simply, it just doesn't feel right. How on earth can simple formulas outperform

11/ human experts, time and time again? It's us and our very human nature that allow this to keep happening. Models beat the human forecasters because they reliably and consistently apply the same criteria time after time. In almost every instance, it is the total reliability

12/ of application of the model that accounts for its superior performance. Models never vary. They are always consistent. They are never moody, never fight with their spouse, are never hung over from a night on the town, and never get bored.

13/ They don’t favor vivid, interesting stories over reams of statistical data. They never take anything personally. They don’t have egos. They’re not out to prove anything. If they were people, they’d be the death of any party.

14/ People, on the other hand, are far more interesting. It’s far more natural to react emotionally or personalize the problem than is to dispassionately review broad statistical occurrences – and so much more fun! It’s much more natural for us to look at the limited set of

15/ our personal experiences and then generalize from this small sample to create a rule of thumb heuristic. We are a bundle of inconsistencies, and although making us interesting, it plays havoc with our ability to successfully invest our money. I believe that we humans are

16/ the constant in this little drama. Against all evidence, we continue to believe that while 'that other person' might be dumb, we're not. We can clearly see all those biases and mistakes others make, but can never look in a mirror to see that we are the same.

17/ It's part of our nature and burned into our DNA. Not convinced? Go over to my colleague @jfc_3_'s page, where he shows that we've changed very little in this regard since the founding of markets and exchanges.

Arbitraging human nature is the last sustainable edge. Use it.