Thread by @spiantado

Did you know that there are psychologists who study "stereotype accuracy"? I've always wondered what the hell, so I've been reading it recently. For the record, it’s exactly as bad as it sounds.

Here’s a thread.

I started with @PsychRabble et al's review paper, "Stereotype accuracy: one of the largest and most replicable effects in all of social psychology." It claims the correlation between stereotypes and reality is stronger than most effects in psychology.
gwern.net/docs/psycholog…

Many papers on "stereotype accuracy" seem to begin by complaining that everyone assumes stereotypes are inaccurate (e.g. the word "stereotype" is *defined* to mean inaccurate) without evidence.

Indeed, academic conservatives like @JonHaidt claim not only that stereotypes are accurate, but that denial of "stereotype accuracy" is one of the left's anti-science positions. The equivalent of anti-vaxxers and young-earth creationists.

So, let’s look at some of the data the “stereotype accuracy” claims are based on.

Jussim et al. cite Ashton & Esses, who studied beliefs about academic performance of nine different groups. People's intuitions got about half the variance in average academic performance. But there are statistically reliable misconceptions too.

For this data, Ashton & Esses artificially shifted participant ratings to equal the true average (so the values reflect only relative differences). I wonder what the data looked like before that. Stereotypes must not be accurate enough without a little guiding hand.

But, unless you're deeply racist, you'll know that ethnicity is NOT the causal factor at play (as opposed to, e.g. environmental or cultural factors). So any stereotype that the differences are *about* ethnicity is wrong from the get-go.

Jussim et al. talk about Kaplowitz, Fisher, & Broman -- a study which looked at just four statistics and showed that people kinda know the numbers and the direction of racial differences.

But people weren't great. For example, white respondents are consistently miscalibrated, e.g. underestimating rates of Black poverty, and overestimating income.

Better research has documented that people are actually terrible--I mean, really terrible--at estimating these group differences. They severely underestimate levels of inequality. @mwkraus
journals.sagepub.com/doi/pdf/10.117…

(There's also cool recent interventions to try to correct people's mistaken beliefs. @bennettcallag)
pnas.org/content/118/38…

Jussim et Al. cite McCauley & Stitt, a tiny, narrow study from the 1970s. This field is so empirically weak that 40 years later, people are still writing about what 69 kids in a junior college in PA thought about Germans. Really.

Jussim et Al. cite Briton & Hall, who studied the well-known stereotypes about men and women with respect to "restless feet and legs", using "um" and "ah", and “decoding skill".

But jeeze, just look at the data. Here's male judgements vs. reality, and female judgements vs. reality.

Why won't liberals talk about these big, important correlations?

That’s R2=0.23, p=0.048 and R2=0.32, p=0.01 --- Yeah stereotypes! Actually, that’s not how Jussim et al report it. Their idea is to report the fraction of correlations that are higher than 0.3 and 0.5. I guess numbers closer to 100% help make the point they want?

To be fair, it’s not all crap correlations. Study 1 from McCauley & Thangavelu shows people know something about the rates men and women have different jobs.

Is the mere knowledge that there are more women nurses than male nurses evidence that “stereotypes are accurate”?

This actually highlights a real demarcation problem for this field: Is any belief about people a stereotype? Vegetarians don’t eat meat? Redheads have hair? Doesn’t the scope of what you include determine whether stereotypes are generally accurate? More on that later.

So this is the stuff that’s on par with the theory of evolution, huh?

My favorite examples, though, are the "national character" stereotypes---a collection of studies that examines people's knowledge of how the "big five" personality characteristics vary across nations. Can people accurately report the level of neuroticism in Germans vs. Czechs?

The answer is no. You can't.

Even Jussim et al. call it "almost completely inaccurate."

They call inaccuracy "a puzzle."

Rogers & Wood do show that 84 Wake Forest undergrads know a little about US regional stereotypes in terms of some (but not all) of the big-five personality characteristics. But...

...even for these it's really unclear *what* they're really about (no controls for state politics, SES, income, population density, etc.).

Though, the raters were all from NC. They, incorrectly, thought NC was by far the most agreeable. Stereotype accuracy?

We could keep going through the studies Jussim reviews, but I think a fair generalization is that it’s basically garbage.

https://twitter.com/MatthewJDalby/status/1417136931746484229?s=20

But there's a much bigger problem with this field than it's data. The entire premise relies on a sneaky trick.

The trick is this: the name--"stereotype accuracy" makes it sound like they are studying stereotypes *in general*. That they have used some method to evaluate the accuracy of people’s beliefs overall and come to a conclusion about your typical stereotype.

But, it’s a fiction. These authors haven’t studied the class of all “stereotypes.” They haven’t randomly sampled stereotypes. As a result, the field has NO ability to evaluate whether the *typical* stereotype is true or not.

It's a field whose primary generalization--”stereotypes tend to be accurate”--is incoherent, because the answer depends on the beliefs you include (redheads have hair?). It’s especially incoherent if you just test a few beliefs while, you know, …

… ignoring *everything else* in history. I wonder if "stereotype accuracy" researchers think that women's suffrage really did ruin men and families? Or were stereotypes not accurate then but are now? What happened?

If stereotypes are typically so great, why do so many horrible claims based only on some aspect of a person's identity need debunking?
splcenter.org/fighting-hate/…

Correcting mistaken stereotypes even had a role in WWII, where the government needed to correct common misperceptions about French allies.

Great title, right?

en.wikipedia.org/wiki/112_Gripe…

Or, remember this guy? Why didn't the “stereotype accuracy” researchers swoop in and measure if Trump supporters’ stereotypes about Mexico (or China or Democrats or ...) were accurate?

(Trump's claims about immigrants are widely believed, but it's well-established that immigrants commit fewer crimes -- by a factor of two or more!
cato.org/publications/i…)

I won't show the heinous examples of stereotypes that have been used against virtually every dimension of human existence: race, gender, nationality, orientation, ethnicity, language/dialect, socioeconomic status, etc.

They're easy to find, and the "stereotype accuracy" people sure know about them. Why aren't the accuracy of these stereotypes in the review, @PsychRabble?

Maybe I will show one historical stereotype, via @SocImages. Have we Irish have changed so much?

Sure--SURELY--researchers have better things to do.

So that's the selection bias that forms this "field": present only work that makes the majority feel better about their prejudices. Don't seek contradictory data. Ignore the quality of the data. Don't test broadly. Be smug about being controversial.

Ignore history.

It’s important to note too that even debates about whether stereotypes *tend* to be true or not are probably beside the point. Here’s @GeorgeTakei

But here’s what puzzles me most. There's p-l-e-n-t-y of really great work that investigates whether people's beliefs are calibrated to reality. It's a foundational question for many cognitive theories.

But, unlike that work, “stereotype accuracy” researchers have intentionally *chosen* to use a term which aligns themselves with some of the worst beliefs humans have had about other humans throughout history.

Instead of generally studying processes and mechanisms of belief formation--so important in the age of covid, Trump, climate, etc--they decided instead to try to rehabilitate everyone’s idea of "stereotypes"...

Why on earth would someone chain themselves to that cinderblock.