,
16 tweets,
4 min read
Read on Twitter
New study of fatal police-involved shootings reported “no overall evidence of anti-Black ... disparities.” @dean_c_knox & I submitted a letter to the editor showing study's approach can't support claim. Journal declined to publish. Sharing here so future research can improve. 1/N 
The study claims its approach “sidesteps the benchmark debate”---the problem of picking a baseline to use to evaluate shooting rates across racial groups. We show this is not true.The study implicitly and wrong assumes black/white civilians encounter police in equal numbers.2/N
Without this unjustifiable assumption, the results in this study are entirely inconclusive. The data does not rule out severe anti-black bias, severe anti-white bias, or no bias at all. Analyses of the role of officer race suffer from the same problem. 3/N
The study claims “racial disparities” in its analysis are “a necessary but not sufficient requirement for the existence of racial biases”—that if no anti-black disparity is found, no anti-black bias exists. 4/N
In an interview, one author claimed the results show "no support for the idea that white officers are biased in shooting black citizens.”5/N msutoday.msu.edu/news/2019/the-…
But to demonstrate racial bias, analysts must show that Pr(shot|civilian race, X) differs by race. The study analyzes Pr(civilian race|shot, X), but makes strong claims about Pr(shot|civilian race, X). 6/N
To see why Pr(civilian race|shot, X) is the wrong quantity, imagine police encounter 100 civilians---10 black and 90 white---in identical circumstances. Due to anti-black bias, they shoot five black civilians (50%), and nine white civilians (10%)... 7/N
Under this hypothetical, the study’s approach would show a much higher chance the victim is white conditional on being shot (9/14 = .64) than black (5/14 = .36), and erroneously conclude no anti-black bias. 8/N
The study invokes the same fallacy when analyzing officer characteristics. Table 2 shows the relationship between Pr(civilian black|shot, officer race, X) and Pr(civilian white|shot, officer race, X) is not significantly different between white and black officers.... 9/N
From this, the study concludes: “white officers are not more likely to shoot minority civilians than non-white officers.” Again, this inference only follows under the strong, unstated assumption that black and white officers encounter black civilians in equal numbers. 10/N
But consider another hypothetical. Suppose black officers encounter 90 black civilians and 10 white, while white officers encounter the reverse. Among these, black and white officers both shoot five black civilians and nine white… 11/N
Clearly, black and white officers in this hypothetical example exhibit very different biases. Examining fatal shootings alone, these biases are entirely concealed. 12/N
Some have told us our argument comes down to a matter of preference. A simple application of Bayes' Rule shows it is a matter of logic. We cannot infer anything about Pr(shot|civilian race, X) by estimating Pr(civilian race|shot, X) without unjustifiable assumptions. 13/N
To be clear, even if the goal is merely to describe the rates of shootings by white/black officers of white/black civilians (i.e., make no claims about racial bias as the cause for observed disparities), the approach in this study is uninformative. 14/N
Studying racial bias using police records requires careful consideration of the unit of analysis, causal estimand, completeness of data and identifying assumptions. Please see our paper w/ @conjugateprior N/N papers.ssrn.com/sol3/papers.cf…
Our comment can be downloaded here: papers.ssrn.com/sol3/papers.cf…
