THREAD: Gelman rightly argues that visualizing the data is critical to making sense of complex analyses/findings like those in this "national IQ" study. It argues that "high IQ" countries may better weather secular trends like decreasing religiosity. Doesn't seem so to me. (1/n)
2/n: There are good reasons to be extremely skeptical of these "national IQ" data. For example, see: sciencedirect.com/science/articl… and sciencedirect.com/science/articl…
3/n: BUT, leaving those major concerns aside, do the data support the authors' claims? Here are the claims/conclusions from their Study 1. Only lower-IQ countries had homicides increase w/secular decrease in religiosity: 
4/n: These conclusions are supported by regression models that, for the life of me, I can't figure out were used over multilevel models. But, others are more expert than I am about that. I just went to their data (posted on OSF: yay!) osf.io/3sf94/
5/n: From Study 1, I retained countries w/at least 1 (of 3) IQ scores and w/at least 2 years of the predictor (religiosity) and outcome (homicide rate at same level of aggregation). Per country, I averaged the 1-3 IQ scores available into an IQ composite (rs = .85-.97).
6/n: I then ranked the countries by "IQ", and selected the bottom 9, the next-to-bottom 9, the next-to-top 9, and the top 9, and for each plotted religiosity & homicide rate by year.
7/n: Remember, the claim is that relig decline from '45-'10 predicted increased homicides in "low IQ" countries only. What did I see? First, MUCH less data available for "low IQ" countries; analyses must've imputed a lot.
8/n: Second, ALMOST NO "lower-IQ" countries actually showed a secular decrease in religiosity! Third, ALMOST NO "higher IQ" countries showed any changes in homicide rates to off the floor. Data figures in next tweets:
9/n: Here are the data for the "bottom IQ" countries at left and the "next-to-bottom IQ" countries at right. Which show the supposed pattern? Arguably only Saint Lucia, Gha, & Dominica, and weakly. (note: 14 countries have only 2 time points!)
10/n: Here are the data from the "next-to-top-IQ" countries at left & "next-to-top-IQ" countries at right. Notice how much more data are available. Almost all countries show secular decrease in religiosity, but homicide rates are also at floor for almost all.
11/n: It seems to me, then, that the regression analyses don't reflect the "ground truth" of the data that went into them. Maybe Study 2 is convincing (I'm done checking), but I don't think that Study 1 should move anyone's priors (even disregarding the Q of national-IQ validity)
12/n: Maybe I've missed something or made errors, so I'm tagging two of the authors on Twitter: @ImHardcory @EPoe187
13/13: Also tagging a few people who commented earlier on some others' tweets about this study: @VandekerckhoveJ @RebeccaSear @hansijzerman
