Thread by @cecilejanssens

This might well be the most amazing project that I have ever worked on: the validation of a new method for searching scientific literature. The first paper is now online for comments.
(thread)

peerj.com/preprints/2764…

A while ago, my students were stuck finding relevant literature for their projects. I had just given myself a break from grant writing and had time to help them out---a decision I will never regret (the opposite, I keep wondering what discoveries we miss by being too busy ...)

Searching scientific literature is time-consuming and, on new topics, full of uncertainty: How do we know that what we found was the best that was to be found? How do we know our search query was appropriate? Often, we don’t.

Ideally, we like a method that finds relevant articles through other articles that we know or found through a quick search. “Related articles” in PubMed, Web of Science, and Scopus have this possibility, but these functions are not great. This is where our new method fits in.

The CoCites method consists of two searches. The core is a co-citation search that finds and ranks articles that are frequently cited together with the ‘query’ articles that you know already. The second is a citation search.

We programmed an online tool for our research project that automatically performs the searches. Here’s how it works.

Suppose we know this article on exercise in pregnant women with diabetes and want to know who else published on this niche topic. We find the article, click “perform co-cited search”, and find 311 articles that are co-cited with the query article more than once.

Here are the top 25 results. Red = exercise, Blue = gestational diabetes.
You see that the results are all highly relevant, but also that the most recent publication is from 2014. Articles that are not cited enough, cannot be co-cited. That’s why we do a citation search next.

As the citation search only works on larger query sets, we perform the search on the top 10 articles (arbitrary, though n should be >10 for best results). We click ‘Cited and citing articles”, and find that 160 articles are cited by or citing 2 or more of the 10 articles.

Here are the top 25 results. Again highly relevant. And all recent papers.

In our study, we applied the method to 250 published systematic reviews. We started a co-citation search with the 2 highest-cited articles in each review, screened the results to find relevant articles, and added these to the query set to run the citation search.

Overall, we retrieved a median of 75% of the articles included in each review. We observed, unsurprisingly, that many reviews were of uncertain quality, comparing apples and oranges. In a (a priori identifiable) subset of reviews, we retrieved 88% of the articles. Not bad.

Overall, the number of articles needed to screen was similar to what the authors in the reviews had done. Similar, only because 40% of the authors had screened <500 articles. CoCites can't beat that ... but was more efficient in the rest of the reviews. Not bad either.

There is more in the paper and more to come soon. Implementation is the goal. Let is know that you are interested: sign up at cocites.com to stay informed. Sharing is appreciated (Librarians, I love to hear your thoughts, please send me a DM).