Thread by @GidMK

Everyone loves dogs, so a study that found that dogs could lower your risk of schizophrenia has been Big News

Unfortunately, I'm here as The Science Grinch to ruin everyone's day

Dogs probably don't stop schizophrenia 🧵

Study is here: journals.plos.org/plosone/articl…

So, what did the scientists do?

Well, on first blush it looks like they did a simple cohort study - took a group of people, split them up by whether they were exposed to cats/dogs, and compared their mental health years later

But then you look a little closer, and you realize that this is actually a CASE-CONTROL study, because the authors selected their patients based on outcomes (mental health diagnosis), not exposure (having pets)

(Tip for non-epi people: the difference between a case-control and cohort can be hard to get at first, but just look at how they selected their participants. If they picked people based on their outcomes, it's case-control)

Ok, so it's a case-control being reported as a cohort study

That's a problem, maybe, but not that much

Let's look at the results

And this is where things get really, REALLY weird

Compare this sentence in the results to the graph. Notice anything strange here?

(Note: I'm not talking about how they used cross-sectional data to fudge longitudinal analyses. We'll get to that)

The sentence I highlighted above is just...wrong

If that's the analysis they did, then they didn't show that "the time of the first household pet dog...was associated with a significantly decreased hazard of having a schizophrenia diagnosis"

This graph shows a Kaplan-Meier curve plotting the proportion who got a dog against the age that they reported getting one, by group

It's testing whether people who are currently diagnosed with SP or BPD were more likely to get a dog as a child!

In other words, the main finding of this paper, the one reported everywhere, is NOT that dogs reduce your risk of schizophrenia, it's that schizophrenia reduces your risk of dogs!

Now, it's not the fault of the media that this was reported wrong - it's wrong IN THE PAPER

This is perhaps not surprising, because there are quite a few other mistakes...

This sentence is statistically weird. If you're correcting for multiple comparisons (good) you can't just undo that by using the 'trend'

The authors also report 'relative risks' even though they used logistic models

You ~can~ convert the results of a logistic model to a relative risk, but it's not that easy and they didn't report doing so, which means the RRs are probably just ORs

But all of this pales in comparison to the massive issue I skipped past earlier

The study used longitudinal analysis (Cox/KM) to analyze cross-sectional data

https://twitter.com/GidMK/status/1220462617472487425?s=20

You'd think from the results that they had monitored children over time, recorded when they got a pet, and then followed up years later to find out if they got a diagnosis

But remember - this is a CASE-CONTROL study

What the authors actually did is ASK PEOPLE WITH A CURRENT DIAGNOSIS when they remembered getting a dog as a child

This is an inherently flawed approach

Imagine asking someone who is currently going through an inpatient admission for acute psychosis when they remember getting a dog as a child, and using that as your exposure variable

You see the problem

So this study didn't even really measure whether people who were diagnosed with schizophrenia were more likely to get dogs as children

It measured who was more likely to REMEMBER getting a dog as a child

And using the time that they remembered having a dog as the time-point for a longitudinal analysis is...problematic

For one thing, you don't have accurate pet death data. You don't actually know how long these people were exposed to pets!

I'm going to stop here, because while there's more there really isn't much point in going through it all

The fact that the analyses were the wrong way around, the exposure was a bit meaningless, and that the stats were probably misreported is probably enough

And yet, the study was published, and got into the NYT

*sigh*

TL:DR

- dogs don't prevent schizophrenia
- schizophrenia may prevent dogs
- even then, probably not
- this study is a mess

If you want more info, @statsepi and @ADAlthousePhD explain really well in this thread:

https://twitter.com/statsepi/status/1220015078726098944?s=20