Thread by @psforscher

In press with @CalvinKLai, @jordanaxt, @CharlieEbersole, Michelle Herman, @DevineLab, and @BrianNosek : "A meta-analysis of procedures to change implicit measures" psyarxiv.com/dv8tu/

This is a huge project & I have lots of thoughts about it. I discuss some below

[THREAD]

Implicit bias is frequently invoked as a way to understanding social problems, especially disparities between social groups. However, social disparities are not the only area: there's a burgeoning literature applying the concepts of implicit bias to addiction

It's often claimed (including by me ncbi.nlm.nih.gov/pubmed/23524616) that changing implicit bias is a means of solving social problems. However, many o fhte studies used to support this claim are observational. We wanted to focus on randomized studies that can get closer to causality

First, some good news. There are lots of procedures to change implicit bias. We identified 11 categories of procedures and many seemed to shift measured implicit bias

However, even though many are interested in implicit bias research *because* it might provide a route to illuninate and address social problems, there are many limitations in the evidence base of randomized studies to change implicit bias

For example:

- 93% of studies did not measure outcomes beyond a single session
- 80% involved university samples
- 83% of studies included no measurement of behavior

If we focus on the 17% of studies that did measure behavior many of the behavioral measures have tenuous links to important social outcomes. For example, one popular outcome is how far a person wishes to sit next to a Black confederate ncbi.nlm.nih.gov/pubmed/21928921

Seating distance *may* be important in the context of an interview, for example, but it's certainly a far cry from something like a police shooting

If we take it as a given that the behaviors assessed in these studies are important, the degree of change in behavior is quite different than the degree of change in implicit bias. Where there's evidence of behavior change, there is no evidence of change in implicit bias

In fact, when we tested whether there's evidence that change in behavior was mediated by change in implicit bias, we found no evidence of this. There was also no between-studies heterogeneity, meaning there's not a subset of studies that shows greater evidence for mediation

Where does this leave us?

At the very least, I think it's fair to say that the evidence on randomized procedures to change implicit bias does not speak strongly to the causes and possible remedies for social problems

I might go even further: I have started to doubt that implicit bias is an important driver of behavior in the first place -- in other words, maybe implicit bias is an epiphenonemon. In our paper we call this the "scar" interpretation of implicit bias

This is not the only viable interpretation of the results. I urge you to read the paper and consult this thread this thread by my co-first-author, @CalvinKLai

https://twitter.com/CalvinKLai/status/1127958418113875969

Finally, I want to especially acknowledge Calvin, who was a full and equal collaborator on this project. This project started as two independent meta-analyses that we merged. We disagreed a lot, but I think the finished product is that much better for our disagreement