Thread by @EpiEllie

New paper out today in @PLOSONE

With @DrCorlin & Amanda Sullivan, we tried to answer the question “how much epidemiological research published in medical journals actually includes epidemiological expertise?”

The answer was more hopeful than I expected!

journals.plos.org/plosone/articl…

What did we do?

First, we selected 15 top journals (based on Scimago rankings) from the categories “epidemiology”, “general medicine”, and “specialty medicine” (5 in each)

For each one, we randomly chose 3 issues from 2000 & 3 from 2010, plus the first available issue in 2020.

Next, we (and by “we” I mean Amanda who did the data collection!) compiled a list of first, last, & corresponding authors of all original research articles in the selected issues.

We wanted to know what they knew, so instead of trying to figure it out from CVs, we asked them—we sent them a description of our study & a link to complete consent & answer a survey.

All-in-all, Amanda contacted the authors of 1240 journal articles!

Sometimes we couldnt reach the corresponding author, so we reached out to the first or last author (whichever wasnt corresponding). But we tried lots of google-sleuthing first, to see if there was any way we could reach those whose emails bounced.

Now, here’s our first big caveat: authors of older articles were harder to find contact info for, but even for more recent research, our response rates were kinda low 😔😔

Also, knowing #epitwitter as well as I do, I wouldnt be surprised in epidemiologists are over-represented in our respondents!

Y’all know we love to help people get complete data!!

With that caveat, here’s what we asked them.

We started with some questions about themselves — asking them to recall the time the paper in question was written. This included age, gender, education level & degrees, country of training.

We also asked them to recall:

“had you ever received formal training in epidemiology? (select any: at least a full semester of an undergraduate-level course, at least a full semester of a graduate-level course, a workshop, an online class, training in another place, no unsure)”

We asked the same question a second time about their biostatistics training since epi and biostats share a lot of overlap!

(We maybe should have asked a third more general stats question, because some added other stats background in the free text space)

Anyone who hadnt had training, was unsure about training, or didnt answer those questions was asked the follow-up qs:

“As of the time your paper was published, would you have benefited from formal training (at least a full semester course) in epidemiology [biostatistics]?”

Everyone was then asked to rate their confidence in their own epidemiological & biostatistical skills (at the time the paper was written) in applying the methods required in the paper: not at all, not very, somewhat, very, or extremely confident

Next, we asked survey respondents to tell us about their team members, including co-authors’ training, role, and abilities.

Specifically, whether they had a epi or biostats coauthor, or a coauthor with some epi/biostats training. We also asked about clinical coauthors.

Then we asked them to rank how confident they felt in their coauthors’ abilities to apply the epi / biostat methods in the paper, and whether they’d had anyone with epi / biostat expertise review the methods.

So that’s what we did. Now what did we find?

Lots of interdisciplinary collaboration!! But with kinda weird fluctuations over time—there’s definitely more work to be done on this topic!

It was a bit of a surprise to find that papers in epidemiology journals had fewer interdisciplinary teams, but this may also be because not all epi papers are clinical and we didnt ask about non-clincial, non-epi/biostat collabs.

We also took a look at how things varied by age & gender of the corresponding author

Key differences:
•younger corresponding authors had more recent training (not surprising!)
•female corresponding authors were more common over time & more common in epi journals

We started this project because of complaints about the quality of epi & biostats methods in clinical papers.

Methodologists often lament that too much work is published without an expert. I thought so too!

But I’d never actually seen any data on this—so we got some!

And this paper provides a reason to hope!

Yes, we may have ended up with a sample of the most epidemiologically-minded authors, so this may be an over-estimate. But the collaboration numbers were fantastically high in the respondent group! Thats some good news!

The specialty clinical journals were worryingly a bit less likely to have epi or biostat collaborators, and this might suggest the need for more targeted out-reach. But even those authors reported quite a lot of collaboration!

There’s more details in the paper & I might be biased but I think it’s a fun read.

Here’s the link again. Go check it out!

journals.plos.org/plosone/articl…