Thread by @lymanstoneky

This study uses a sample with zero causal inference or time variation and a total of 192 women in the dependent variable category of interest to claim that "sexist religious institutions" (i.e. male clergy) have extremely large negative health effects. asanet.org/sites/default/…

They find, I am not joking, that women who set foot in a "sexist institution" (i.e. a church where women can't be the pastor) JUST ONE TIME causes the ENTIRE HEALTH BENEFIT of church attendance to spontaneously vanish.

that is to say, they find that "sexist" religious institutions are associated with lower self-rated health among women, but not as much for men, but that this effect occurs equally among frequent and infrequent attenders.

Do they have any control for selection?

Of course not!

Do they explore the possibility that propositional beliefs in a church might alter attendance rates, which have an independent positive effect?

Nope!

When they finally discuss selection bias in their conclusion, it's laughably sparse, and basically the stats equivalent of, "well, that's just, like, your opinion, man"

Within two paragraphs they say "First, our data are cross sectional, so we cannot infer causality" and then turn around and say "Even in
religious congregations, where engagement has well-known health benefits, structural sexism can undermine women’s well-being"

So they're like, "our effects aren't causal except in the cases where we choose to interpret them as causal."

But seriously when I saw that they had dealt with attendance by just assuming it was exogenous (and coding it continuously when it's not a continuous variable! yikes!) .... y'all.

If you find that a church having the rule "Elders must be male" has extremely large negative health effects on people who attended that church 3 times in a year and the exact same effects among people who attended 40 times in a year.... that's weird!

To be frank, that's almost prima facie evidence that it's *not* the church driving the effect. It strongly suggests that there's some other factor at work.

Like, what if you use non-attendees who live in close proximity to the churches in question as a control group? Is it possible this is just a spatial thing? idk

And the fact that, even for women in the most "sexist" churches, ***higher attendance predicted better health***, reeeeaaaally complicates the story.

Now look, I'll be honest.

I'm actually skeptical that attending church has big health effects either way. I think the authors actually treat the past literature on positive effects too credulously!

I mean point blank attendance at church clearly *selects for* healthier people, as attendance at *anything* selects for healthier people, since *sick people just go out less*.

I just can't imagine publishing in a disciplinary flagship journal without, ya know, at least making a good-faith-effort at addressing selection.

Also, can't decide if it bothers me that they appear to have just used a linear regression for a count outcome scaled 1-4. Feel like Poisson would be more appropriate but it wouldn't really impact effects much. But I'm also not sure we should care much about 2s and 3s.

Like, my gut is that what you actually want to do is do some kind of binary distribution to predict odds of selecting the highest/lowest SRH categories because I just don't trust that people reliably assign mid-grades.

The mean respondent gives themselves a 3 on a scale of 1 to 4, so I suspect that *4* is actually the modal response.

But this is all just nitpicking that probably doesn't change results.

but seriously don't code religious attendance as continuous it's a factor variable

if you MUST code it as continuous then at least convert to like a frequency index that actually scales rationally