Neal R Haddaway Profile picture
Apr 17 • 8 tweets • 4 min read
🚨greylitsearcher update🚨

Significant upgrade to greylitsearcher! Now supports advanced searching of Google AND provides a search report text file.

estech.shinyapps.io/greylitsearche…

Here's some details... 🧵

#medlibs #InformationRetrieval @SystematicSearching #EvidenceSynthesis greylitsearcher hex logo
You can now build complex Google site: searches the same way you can in Google Scholar... screenshot of the search pa...
And you can search across multiple sites simultaneously... (just be sure they support Google site: searching)

developers.google.com/search/docs/ad… Screenshot showing filled i...
greylitsearcher builds a set of links for each page of search results for each site you enter: Screenshot of the results t...
It then grabs the HTML for each page of results, pausing to remain 'friendly' to the bots at Google HQ: Image
The grabbed HTMLs are then scraped for patterned data and the results are tabulated for each source page, showing titles, descriptions and links: Image
As well as downloading the CSV file of search results, you can save a search history record, showing exactly what you searched for and when, and which URLs were grabbed and scraped. search record text file sho...
If you have any feedback or troubles using greylitsearcher, please let me know by adding an issue on the GitHub repository here:
github.com/nealhaddaway/g…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Neal R Haddaway

Neal R Haddaway Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @nealhaddaway

May 5
Are you attending the #Stockholm50 academic conference on June 1st? Look out for my photography exhibition, 'Hope? and how to grieve for the planet'.

#Photography #DocumentaryPhotography #PhotoStory #ClimateCrisis #ClimateChange #ClimateEmotions

nealhaddaway.com/hope

(1/4)
My project asks how environmental scientists and communicators deal with the emotional impact of working with alongside #EnvironmentalCrises.
nealhaddaway.com/hope

Many of us a grieving for our planet, and sharing our #emotions could be vital in the continuing battle.

(2/4) Black and white photographs...
I'm really honoured to have my work selected alongside several other arts projects aiming to complement the academic conference.

If you're in Stockholm (01/06/22) and want to come, send me a DM for a (free) registration link.

#Stockholm50

(3/4) "Hope? and how to grie...
Read 4 tweets
Oct 12, 2020
In this paper in @NatureEcoEvo we outline 8 major problems that can occur with traditional ways of reviewing the literature, and provide concrete advice on how to avoid them: rdcu.be/b8pp0

This thread outlines our key points! (1/23)
#EvidenceSynthesis #SystematicReview
Traditional ways of reviewing the literature may be susceptible to bias and end up giving us incorrect conclusions.

This is of particular concern when reviews address key policy- and practice- relevant questions. (2/23)
Systematic reviews aim to maximise rigour and minimise susceptibility to bias using rigorous methods.

But despite these methods being available, poor reviews are still published: these could be improved with a few key processes; some not prohibitively costly. (3/23)
Read 23 tweets
Oct 9, 2020
Check out our new paper on the need for better understanding of academic searching:
onlinelibrary.wiley.com/doi/10.1002/jr…

#InformationRetrieval #MedLibs #EvidenceSynthesis

A thread with our key points... (1/20) Image
We agree with others that we now face an 'information crisis' (#infodemic). There is SO much published research we need to find and digest.

Doing this reliably requires systematic review approaches, but even then, it's hugely challenging to find all relevant research.
(2/20) Image
Academic searching/information retrieval is an art form, and there is no 'perfect' search strategy - it takes careful planning and requires substantial skill and training.

These searches are highly complex and must be used in fit-for-purpose bibliographic databases. (3/20) Image
Read 20 tweets
Oct 8, 2020
I've built an R package to scrape search results from up to 100 pages of hits from Google Scholar: github.com/nealhaddaway/G…

It's probably a bit buggy, but feedback greatly appreciated!

Here's a thread on how it works...

#AcademicTwitter #InformationRetrieval #MedLibs
1) It generates a set of up to 100 URLs, each corresponding to a page of visible search results Image
2) It then downloads each of these pages of results as an HTML file Image
Read 6 tweets
Apr 24, 2019
Here's a thread on what's very wrong about using only Google Scholar for a literature view, as done by Buribalova et al. in this recent review: onlinelibrary.wiley.com/doi/full/10.11… #SystematicReview #EvidenceSynthesis
Problem 1: Their search string is flawed. The authors say they used a 'systematic' search for articles on Google Scholar. However, their search will not work as intended because:
a) GS doesn't support Boolean operators (AND, OR, NOT); b) you cannot nest more than one substring (bracketed set of synonyms) in a search; c) they have not nested geographical synonyms within brackets...
Read 15 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(