greylitsearcher builds a set of links for each page of search results for each site you enter:
It then grabs the HTML for each page of results, pausing to remain 'friendly' to the bots at Google HQ:
The grabbed HTMLs are then scraped for patterned data and the results are tabulated for each source page, showing titles, descriptions and links:
As well as downloading the CSV file of search results, you can save a search history record, showing exactly what you searched for and when, and which URLs were grabbed and scraped.
If you have any feedback or troubles using greylitsearcher, please let me know by adding an issue on the GitHub repository here: github.com/nealhaddaway/g…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
It's #WorldMentalHealthDay - so I want to spend some time talking about what I call 'gnostalgia'.
From the Greek ‘gnosis’ (knowledge) and ‘algos’ (pain) - this word captures how it feels to understand the science behind our planetary crises.
#climateEmotions
A 🧵 1/16
Many researchers will feel unease at discussing emotions in a professional context - subjectivity is even trained out of us.
But no human is truly objective.
Seeing our planet destroyed and its people suffer has an immeasurable toll.
#climateCrisis #planetaryCrises
2/16
But not talking about the emotions we face can leave us feeling isolated and alone, unaware that many if not most of our colleagues feel similar emotions.
No, it's not an exciting time, it's terrifying - we're sleep walking into the end of civilisation as we know it, sat back on our soft sofas, drinking lattes, watching Netflix as the vast majority of the planet's population deals with all the crap we've done to get us here. (2/7)
Even as our lives get harder we can't even imagine how shit things are going to be for the less privileged 90% of the human race. I watch heroes getting arrested for desperately trying to get politicians to wake up and act urgently, whilst middle-aged white men hurl abuse.. (3/7)
In this paper in @NatureEcoEvo we outline 8 major problems that can occur with traditional ways of reviewing the literature, and provide concrete advice on how to avoid them: rdcu.be/b8pp0
Traditional ways of reviewing the literature may be susceptible to bias and end up giving us incorrect conclusions.
This is of particular concern when reviews address key policy- and practice- relevant questions. (2/23)
Systematic reviews aim to maximise rigour and minimise susceptibility to bias using rigorous methods.
But despite these methods being available, poor reviews are still published: these could be improved with a few key processes; some not prohibitively costly. (3/23)
We agree with others that we now face an 'information crisis' (#infodemic). There is SO much published research we need to find and digest.
Doing this reliably requires systematic review approaches, but even then, it's hugely challenging to find all relevant research.
(2/20)
Academic searching/information retrieval is an art form, and there is no 'perfect' search strategy - it takes careful planning and requires substantial skill and training.
These searches are highly complex and must be used in fit-for-purpose bibliographic databases. (3/20)