Thread by @autoregress

I just posted a new working paper, with David Arnold (@PrincetonEcon) and Will Dobbie (@Kennedy_School)

It's called “Measuring Racial Discrimination in Bail Decisions”

bit.ly/2QWz4Cu

Here's a short summary thread 👇

Racial disparities are pervasive in the criminal justice system. But do they reflect racial discrimination, or unobserved differences in criminal behavior?

We develop new quasi-experimental methods to answer this question in the context of bail decisions

In most pretrial systems, judges decide whether to detain defendants after a brief case review. They seek to minimize the risk of future crime or failure to appear in court

In NYC, black defendants are 5pp less likely to be released than observably similar white defendants

We define discrimination by how much a judge is less likely to release black defendants than white defendants w/ identical misconduct potential Y*

The issue is that Y* is unobserved and so cannot be conditioned on

This can lead to omitted variables bias in observed disparities

We overcome OVB with two observations

1) When judges are quasi-randomly assigned, OVB is a function of average misconduct risk in the white & black populations

2) Recent approaches to ATE estimation with many discrete instruments can be used to estimate these key risk inputs

I won't dwell on the #metrics details here, but suffice to say I think they're pretty neat

We extrapolate local judge IV variation to estimate ATEs (w/o monotonicity!) and use these ATEs to rescale observed disparities

This yields a measure of discrimination for each judge

What do we find?

- Most (68%) of the observed NYC release rate disparity is due to racial discrimination and not OVB

- Judges differ significantly in unwarranted disparity but most (88%) discriminate against black defendants

- Discrimination correlates with judge observables

We then ask what drives the racial discrimination. Two possibilities:

1. Racial bias (animus or stereotypes)
2. Statistical discrimination

We develop a new hierarchical marginal treatment effects model to separate these channels

I'll again skip the details but there's some neat #metrics here too

We avoid usual IV monotonicity by specifying a *distribution* of MTE curves across judges and races

This allows for variation in judge skill ("signal quality"), key to one form of statistical discrimination

We find both racial bias and statistical discrimination in NYC

Bias & statistical discrimination due to mean risk differences tends to hurt black defendants

Statistical discrimination due to signal quality differences tends to help black defendants (on average)

Finally, we ask what this might mean for policy

We use the model to simulate policies that target + correct the release rates of judges w/ high unwarranted or observed disparities

Both targets appear to reduce average discrimination, despite estimation error and OVB

Why? It turns out that observed racial disparities are super predictive of racial discrimination, despite OVB

This is reminiscent of findings for observational value-added models in the education setting. E.g. bit.ly/2wLlJ9a and bit.ly/2VZejta

There's a lot more in the paper, and we'd love any comments (please DM or email!)

We hope these methods may prove useful in other settings, where:

1. Discrimination is a concern
2. Randomized audit studies are infeasible
3. Decision-makers are quasi-randomly assigned

/thread