Thread by @Epi_D_Nique

Since selection/collider bias is currently a hot topic, it seems like a good time for a #tweetorial!

Gather round #EpiTwitter (but not too close!) for a tour of my paper investigating the when's and why's of selection bias in birth defect epi. 👇🏻

1/🤷🏼‍♀️

onlinelibrary.wiley.com/doi/10.1111/pp…

One of the reasons reproductive epidemiology is trick is that you are *always* dealing with selection.

Who becomes pregnant? Who stays pregnant? Who delivers too soon? Who has an induction?

Is the population in your study a biased sample?

2/
link.springer.com/article/10.100…

Let's add in a rare outcome with a strong effect on survival: birth defects.

Because specific birth defects are rare, we need huge samples to do meaningful studies. But many times data sources with these properties only have live births.

Can we still do valid studies?

3/

This is the question that we tried to answer!

But not just "can we do valid studies?" But also "WHEN can we do valid studies?"

And so was born this paper, currently out ahead of print @PPE_Journal!

4/

Let's start out with why we are concerned about selection in birth defect studies in particular.

Say we are looking at a medication.

That medication could cause a birth defect which may decrease the probability of being born alive *AND* may independently affect survival.

5/

If we include all individuals with the birth defect regardless of whether they survive to live birth, we're good. 👍🏻

But if we only include survivors to live birth, well, now we are stratifying on a collider, which may lead to collider stratification (aka selection) bias. 😬

6/

But not all exposures affect survival through a path other than the birth defect, and not all birth defects impair survival dramatically.

Maybe we can erase some of those arrows and voila! No selection bias!

But maybe we can't - how far off is our answer then?

7/

Since data on who is missing is... uh... missing, we used a study that included all identified birth defect cases (live births, stillbirths, & elective terminations) and then induced selection bias by restricting our analysis to just live births & compared results.

8/

Since we want to know *when* serious selection bias happens, we needed to test different scenarios.

First, we picked birth defects with different effects on survival for analysis (% live births):

Cleft palate (98%), spina bifida (88%) , omphalocele (85%) anencephaly (33%)

9/

Then we picked exposures that had different strengths of association with birth defects and with survival independent of birth defects:

Smoking (moderate for both), antiepileptic drugs (strong for BD, low for survival), and multifetal pregnancy (moderate for both)

10/

Still with me?

I know, it's a lot. But hang in there - it gets interesting!

11/

To see how strong the relationships are between live birth among our chosen birth defects and exposures, we did crude logistic regressions of live birth vs non-live birth (stillbirth or termination).

Here's what we found👇🏻

Look at omphalocele & anencephaly! 👀

12/

So! We definitely have some strong relationships between our exposures and survival among our cases. Keep those arrows.

We couldn't look at this for controls bc this study only has live-born controls. 😱

I know... I know... But it's fine. Trust me. We'll come back to why.

13/

Since we didn't have controls, we had to get creative to examine birth defect ➡️ live birth. 😬

Cleft palate became the reference (survival apx no BD)

Anencephaly (OR 0.007, 95% CI 0.004, 0.01), omphalocele (OR 0.09, 0.06, 0.15), and spina bifida (OR 0.10, 0.06, 0.15)

14/

The results!

What did we find?

Not quite what we expected...

The estimates for most situations were very similar when we included everyone or restricted to live births. 🤔🧐

But we see clear selection bias in that top-right corner: the estimate drops by half! 😳

15/

Interestingly, we see a *decreased* risk of anencephaly with smoking, particularly among live births. Sound familiar?

That the "protective effect" decreases as non-live births are added suggests that this is due to collider/selection bias. Early losses prob. contribute too.

16/

That the results for cleft palate are the same all around is not surprising; after all, 98% of cleft palate cases are live born!

So that pretty much removes the arrow from birth defect to live birth. We no longer have a collider, therefore no more collider/selection bias.👍🏻

17/

But in other cases we seem to meet the criteria, yet estimates are essentially the same.

There's a strong association between omphalocele & live birth (OR=0.09)

&

For live birth by exposure (OR=2.2 for smoking & 5.2 for multiples) among omphalocele cases

So what gives?
18/

Well... We did some digging and found something kind of neat:

It's not the prevalence of exposure by pregnancy outcome that mattered (left columns), but rather the prevalence by all included outcomes (right columns)!

Which... kind of makes sense.

19/

Zooming in on omphalocele, although there are big differences in prevalence among the groups, adding each pregnancy outcome to live births doesn't change the prevalence (& thus the comparison to the reference) much.

Live births are the majority, so they drive prevalence.

20/

Whereas, we see pretty big changes to the prevalence for anencephaly when the exposure is mutlifetal pregnancies and moderate changes for smoking.

Remember, only 33% of anencephaly cases are live born.🤔

21/

I had to ponder on this for a while before I came up with a hypothesis:

Just like controls need to be representative of the population from which the cases arose, so do the cases!

Or rather, the exposure distribution in the cases in your strata needs to be representative.

22/

So when are you most likely to have an unrepresentative case strata?

When you select a small portion of cases in which to examine your question of interest.

In the larger strata, results should be pretty unbiased - certainly within measurement error range.

23/

And as you stratify (aka adjust) for more things, the smaller and weirder strata get, the less representative the exposure distribution will be, the more likely strong and misleading selection bias is likely to be.

This is why adjusting for everything can cause trouble.

24/

So when can we investigate possible causes of birth defects with just live-born cases?

For birth defects where most cases are live born, it appears we are very likely to get good estimates even in seemingly dicy situations. But it's best to exclude highly fatal defects.

FIN!

Addendum:

Why doesn't it matter that we only have live born controls?

Doesn't the population from which the cases arose include normally formed fetuses who didn't survive to live birth?

YES! It's just that they're *such* a small portion that it's NBD.

Don't believe me? 👇🏻

Since neither you (nor the reviewers) felt that "trust me" was a sufficient response, we did a quantitative bias analysis to see what the OR of exposure & non-live birth for controls would need to be to bias results.

Answer: absurdly high.

Filled diamonds ~ our study results.

And very very finally, the vital stats!

This paper took more revisions than I can count!

Desk reject = 1
Reject after review = 1
Major revisions = 1
Minor revisions = 1

Time from first sub to publication: 1 yr 3 months

And that doesn't count the 5 million internal revisions.

Cc: @NBDPN