Neal R Haddaway Profile picture
Photojournalist and documentary photographer | Environment and climate research | Evidence synthesis methods trainer

Apr 17, 2022, 8 tweets

🚨greylitsearcher update🚨

Significant upgrade to greylitsearcher! Now supports advanced searching of Google AND provides a search report text file.

estech.shinyapps.io/greylitsearche…

Here's some details... 🧵

#medlibs #InformationRetrieval @SystematicSearching #EvidenceSynthesis

You can now build complex Google site: searches the same way you can in Google Scholar...

And you can search across multiple sites simultaneously... (just be sure they support Google site: searching)

developers.google.com/search/docs/ad…

greylitsearcher builds a set of links for each page of search results for each site you enter:

It then grabs the HTML for each page of results, pausing to remain 'friendly' to the bots at Google HQ:

The grabbed HTMLs are then scraped for patterned data and the results are tabulated for each source page, showing titles, descriptions and links:

As well as downloading the CSV file of search results, you can save a search history record, showing exactly what you searched for and when, and which URLs were grabbed and scraped.

If you have any feedback or troubles using greylitsearcher, please let me know by adding an issue on the GitHub repository here:
github.com/nealhaddaway/g…

Share this Scrolly Tale with your friends.

A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.

Keep scrolling