Forthcoming @The_JOP: psyarxiv.com/49hfg. We re-analyzed data of 7 published studies and conducted large-scale replications with original authors of the seminal study. Findings raise significant concerns about past use psychophysiology in political science and beyond [1/11] 
The seminal study in @sciencemagazine found that conservatives react with higher EDA to threat pics: science.sciencemag.org/content/321/58…. Follow-up studies celebrate EDA as an efficient behavioural measure of arousal. Yet, no attempts in poli sci has been done to validate measures. [2/11]
We conducted replications with two of the original authors in Aarhus, Denmark and Lincoln, Nebraska. We also assessed EDA reactions to disgusting, positive and neutral pictures. Results were mixed: No association of threat EDA and ideology in Denmark. The question: Why? [3/11]
One possibility: The association between feelings of threat and ideology is particular for the US. But we found that self-reported threat was equally strongly related to social conservatism in both Denmark and United States. So, something else must be going on. [4/11]
Another possibility: The association is fickle because the measure of threat-sensitivity is noisy. Yes, alphas were .11 and .14 in DK and US! Also, r=.97 between stimulus and baseline levels of EDA (i.e., at best only very small effects). The measure is basically useless. [5/11]
Is this a problem of just our data? No. We collected data from 7 published studies including the seminal study. Only two studies (and not the original Science study) fared better in terms of measurement. These two studies used a method to remove signal drift. [6/11]
Did the measures in these two studies then predict political ideology? No, they didn't. In fact, only the data from two publications showed significant correlations with social conservatism. The original study - and a follow-up publication using the *same* data. [7/11]
Do the filtered, reliable measures even measure threat-sensitivity? We do not find any evidence of that. Instead, they tap a general individual difference across image types. Seemingly, it tracks physio changes associated with changing the attention towards any image. [8/11]
So, what to conclude? (1) EDA responses to threats are not associated with political ideology; (2) this doesn't necessarily reflect that threat responses are not related to ideology. The measures used from 2008 and until now simply do not measure threat-sensitivity. [9/11]
What then? We offer 3 recommendations: 1) Make designs that allows you to test the reliability and validity of your measures. 2) Always collect other, more reliable measures. 3) The current designs do not work - improve facilities, protocols and processing. [10/11]
Finally, a personal note to ECRs: Show maximum caution with psychophys until these issues have been solved. Resources and careers can disappear into this black hole. This null finding costed us about $80,000, 4 years of postdoc salary and more headaches than I can count. [11/11]
