Check out our new paper on the need for better understanding of academic searching:…

#InformationRetrieval #MedLibs #EvidenceSynthesis

A thread with our key points... (1/20) Image
We agree with others that we now face an 'information crisis' (#infodemic). There is SO much published research we need to find and digest.

Doing this reliably requires systematic review approaches, but even then, it's hugely challenging to find all relevant research.
(2/20) Image
Academic searching/information retrieval is an art form, and there is no 'perfect' search strategy - it takes careful planning and requires substantial skill and training.

These searches are highly complex and must be used in fit-for-purpose bibliographic databases. (3/20) Image
We introduce an updated typology of searching that consists of three main types of searches that we academics routinely engage in;

🔎 'Lookup' searching

🔎 'Exploratory' searching

🔎 'Systematic' searching

🔎 'Lookup searches' - also called 'known item searches' or 'navigational searches' - are conducted with a clear goal in mind and "yield precise results with minimal need for result set examination and item comparison."…
Here, the goal is to find specific information quickly and efficiently; for example, finding a paper we know should exist, or any highly relevant information on a targeted topic.

The goal here is NOT to be comprehensive, nor improve understanding of the evidence base.
🔎 'Exploratory searches' - are used to better understand the nature of a topic, involving continued learning by identifying many, sometimes contradicting, knowledge sources to build a mental model on a topic. Our search terms evolve as we go.…
Exploratory searches are an integral part of developing a search strategy for a systematic review or (meta-analysis) - they help teach us about an evidence base and how it has been described and catalogued. It is also known as 'scoping'.
🔎 'Systematic searches' - are unique because they aim to: 1) identify all (available) relevant records in a
2) transparent and
3) reproducible manner
There should ideally be no learning by this stage and the search strategy should be unchanged from the protocol.
We describe each of these searches in our paper, and what characteristics and practices are integral to each - in general there is poor 'search literacy' in the research community that results in inappropriate matching of search goals and search systems.
For example - in this (not so) 'systematic' review, the authors conducted searches only in Google Scholar (…), a search system designed for lookup searches, and one NOT suitable for systematic searches (…)
Many people misunderstand what systematic searches should and should not entail, leading to criticisms of the systematic review methods.
e.g. that they should not entail 'hermeneutic circles' of iterative learning and are thus too rigid.…
In practice, however, systematic searches should always be preceded by a thorough exploratory search phase, which in systematic reviews is called 'scoping'.…
We introduce a 'search triangle' to help clarify this concept. Efficient and effective searching only works when all three concepts (search goals, search systems, and search heuristics) are matched well.
(14/20) Image
We make three important calls to action regarding academic searching that aim to avoid inappropriate use of search systems that weren't designed for rigorous exploratory or systematic searching.
1) We need better awareness of the intricacies of academic searching and acceptance of the importance of matching goals (the 'why'), systems (the 'what'), and heuristics (the 'how') in all academic searching.
2) We need better search education, aiming towards 'search literacy' as the norm.

This requires an appreciation of information science as a vital research skill, and the establishment of training across all research institutions.
3) We need fit‐for‐purpose search systems, designed to support each of the three search types.

Systems designed for rapid lookup are not fit for systematic searches. Systems designed for systematic searches need to facilitate bulk downloads and be fully transparent.
Researchers should put pressure on search system providers to tailor their products to our needs. But to do that, we need to know and accept what those needs are in the first place.
Finally - algorithms (e.g. Google Scholar's) cherry pick evidence for us and we may have no idea it's happening.

If we don't urgently improve understanding of academic searches, we risk being biased and influenced by what system developers think we want to see.

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Neal Haddaway

Neal Haddaway Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @nealhaddaway

8 Oct
I've built an R package to scrape search results from up to 100 pages of hits from Google Scholar:…

It's probably a bit buggy, but feedback greatly appreciated!

Here's a thread on how it works...

#AcademicTwitter #InformationRetrieval #MedLibs
1) It generates a set of up to 100 URLs, each corresponding to a page of visible search results Image
2) It then downloads each of these pages of results as an HTML file Image
Read 6 tweets
24 Apr 19
Here's a thread on what's very wrong about using only Google Scholar for a literature view, as done by Buribalova et al. in this recent review:… #SystematicReview #EvidenceSynthesis
Problem 1: Their search string is flawed. The authors say they used a 'systematic' search for articles on Google Scholar. However, their search will not work as intended because:
a) GS doesn't support Boolean operators (AND, OR, NOT); b) you cannot nest more than one substring (bracketed set of synonyms) in a search; c) they have not nested geographical synonyms within brackets...
Read 15 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!