Collaborative experiments w/ gov. agencies are the gold standard in policy research, but are collaborating agencies representative? New working paper evaluates selection into research partnerships w/ police. Seeking feedback! 1/n @smgoerger @seanjwestwood papers.ssrn.com/sol3/papers.cf… 
We sent ~3,000 local police chiefs/sheriffs in 48 states *sincere* invitations to discuss a potential collaboration (no deception). Two goals. 1) Assess the correlates of willingness to collaborate. 2) Assess the role of agency reputation in collaboration decisions. 2/n
First, are collaborating agencies representative of agencies at large? To find out, we merged responses w/ local data on crime, partisanship, demographics, police use of force and more. But agencies receptive to our offer looked pretty much the same as agencies that declined. 3/n
If agency attributes are not the story, what is? Theory: outside analysts could discover subpar performance---even misconduct---causing reputational harm in the short term. So agencies avoid collaborations because of associated political risks. 4/n
We test this idea w/ a pilot in NJ & pre-registered nationwide replication/extension. One random set of agencies got an offer to discuss a research partnership. Another got additional language: their agency's rank on a performance metric (uses of force, or clearance rates). 5/n
Hypothesis: priming agencies to think about performance evals (and by extension, potential reputational harm) would depress affirmative responses. Also tested impact of confidentiality offers. Both times, performance eval cue caused ~8 point drop in affirmative response rate. 6/n
We also hypothesized treatment effects would vary with agency performance. Being told your agency is doing well should not cause anyone to recoil, right? Wrong. Across most performance ranks, the intervention depressed willingness to discuss collaborations. 7/n
We also wondered whether police considered us liberal academics who couldn't be trusted & dismissed us on those grounds. If so, agencies in more conservative areas may respond more negatively. But Trump's county vote% doesn't predict response, or condition treatment effects. 8/n
Police agencies open to research partnerships appear largely representative. Great news re: external validity of collaborations! But willingness to collaborate is divorced from agency performance. Reputational concerns, not need for improvement, govern these choices. 9/n
Epilogue: we are currently in the process of following up with the more than 300 agencies who showed interest in collaboration to discuss research partnerships. Stay tuned! n/n
