Profile picture
Shawn M. Jones
@shawnmjones
, 18 tweets, 27 min read Read on Twitter
In addition to the excellent tools from the tweeted picture, there are many #webarchiving tools in development by @WebSciDL and the @LosAlamosNatLab Research Library Prototeam.
github.com/oduwsdl
github.com/mementoweb

Some examples below #eResearchNZ19 #DH #eResearch
Ever wonder which archives have a capture of a given URL? What about for a specific time? Do you want to search multiple #webarchives at once?

The Memento TimeTravel service run by @LosAlamosNatLab uses the #Memento protocol for just that: timetravel.mementoweb.org

#webarchiving
From your own machine, MemGator, by @ibnesayeed and @machawk1, also allows one to search across #webarchives using the #Memento protocol.

github.com/oduwsdl/MemGat…

ieee-tcdl.org/Bulletin/v13n1…

#webarchiving
MementoEmbed produces a social card or thumbnail of a #memento from #webarchives for inclusion in blog posts and other web pages. It is by @shawnmjones, with contributions by @ibnesayeed and @machawk1.

github.com/oduwsdl/Mement…

ws-dl.blogspot.com/2018/08/2018-0…

#webarchiving
For seamlessly browsing the web of the past, I use the Chrome web extension developed by @hvdsomp @hariharshankar from @LosAlamosNatLab.

bit.ly/memento-for-ch…

Just pick a date and browse the web as if it were that date!

#webarchiving
ArchiveNow saves a live web resource to multiple #webarchives at once! It is by @maturban1, with contributions by @machawk1, @ibnesayeed, @ruebot, and myano.

github.com/oduwsdl/archiv…

ws-dl.blogspot.com/2017/02/2017-0…

#webarchiving
InterPlanetary Wayback (IPWB) disseminates WARCs via the distributed web Interplanetary Filesystem (IPFS). It is a project by @machawk1 and @ibnesayeed.

github.com/oduwsdl/ipwb

slideshare.net/ibnesayeed/int…

#webarchiving
CarbonDate allows one to find the earliest existence of a webpage using #webarchives and other services, by @hanysalaheldeen, @acnwala, @grantcatkins, with contributions by DarkAngelZT, @ibnesayeed, and @machawk1

github.com/oduwsdl/Carbon…

ws-dl.blogspot.com/2017/09/2017-0…

#webarchiving
Archive-It Utilities (AIU), by @shawnmjones, extracts metadata from @archiveitorg collections.

github.com/oduwsdl/archiv…

ws-dl.blogspot.com/2018/07/2018-0…

#webarchiving
TMVis provides a visualization of thumbnails from #webarchives, allowing users to see changes over time. It is based on a paper by @aalsum, and work by @machawk1, @mir_smi, and @mrgunn.

github.com/oduwsdl/tmvis

ws-dl.blogspot.com/2017/10/2017-1…

#webarchiving
MementoDamage, a service for quantifying #memento quality, is based on research by @justinfbrunelle. The code was developed by @erikaris, with contributions by @ibnesayeed, @machawk1, @grantcatkins, and soedomoto.

github.com/oduwsdl/web-me…

ws-dl.blogspot.com/2017/11/2017-1…

#webarchiving
The Off-Topic-Memento-Toolkit (OTMT) detects soft-404 pages and other off-topic pages in a web archive collection. It is based on work by @yasmina_anwar with code and further research by @shawnmjones.

github.com/oduwsdl/off-to…

doi.org/10.17605/OSF.I…

#webarchiving
.@machawk1 has created #webarchiving Chrome extensions:
* Create WARCs from Chrome with WARCreate: warcreate.com
* Find archived versions of your current page, and/or save the current page to multiple web archives with Mink: chrome.google.com/webstore/detai…

#webarchiving
.@WebSciDL also has Twitter services, like @_wdill and @icanhazmemento run by @acnwala.

@_wdill shows web page evolution over time: whatdiditlooklike.mementoweb.org

@icanhazmemento allows one to save a link to #webarchives from within Twitter!
ws-dl.blogspot.com/2015/07/2015-0…

#webarchiving
.@machawk1 and @johnaberlin have created WAIL, allowing one to build their own personal #webarchives.

github.com/machawk1/wail

github.com/n0tan3rd/wail

dl.acm.org/citation.cfm?i…

#webarchiving
A presentation on the @WebSciDL #webarchiving tools was given at WADL as part of @jcdl2018 by @weiglemc:
slideshare.net/mweigle/enabli…

She also wrote about some of the tools for @SSRC_org: parameters.ssrc.org/2018/09/on-the…
This is not a comprehensive list of tools (or even @WebSciDL projects), but I hope some find this list to be useful. Many of these tools address use cases that most people didn't know that they had. I hope one or more finds its way into your #webarchiving toolkit.
I also use AUT (archivesunleashed.org/aut/) by @unleasharchives, warcio (pypi.org/project/warcio/) by @IlyaKreymer, and @webrecorder_io. I have not tried the new release of #Heritrix yet:

If you know of other #webarchiving tools I should try, please comment.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Shawn M. Jones
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Related hashtags

#webarchiving #eResearchNZ19 #DH #eResearch #webarchives #memento #Heritrix

Related threads

Profile picture
Jenn Strater 🥑
@codeJENNerator
There are just over two weeks left to submit a talk for #GR8Conf EU 2019 at cfp.gr8conf.org

If you need ideas, here's a thread.

#groovylang #grailsfw #gradle #spockfw #gebish #griffon #springboot #micronautfw #ratpackweb #sdkman #android #devops #ci #cd #cfp #jenkins
Feel free to like/❤ a topic you’d attend and @ people you think should submit a talk on this topic. I have A LOT of ideas so I’ll spread this out over a few days maybe a week depending on how this goes.
1/ an intro to @spockframework

including:
- using #spockfw in a polyglot organization including any tips/tricks for integrating with other languages like #Java and #Kotlin
- the top things you wish you knew when you started using spock
- good testing practices in general
Read 15 tweets
Profile picture
Matthew Willemsen
@mwill_crypto
0/ The level of daily activity that takes place in the #crypto and #blockchain space never ceases to amaze me.

Below, a 22-part thread full of project announcements and other ecosystem events from the past ~24 hours. Enjoy!
1/ 🚀 @DharmaProtocol- and @CompoundFinance-powered @Bloqboard introduced its 'Offers to Lend' feature!

Now, lenders can offer to lend $BAT, $DAI, $REP, $WETH and $ZRX in a P2P-manner from their @MetaMask_io wallet. Can be done in under a minute!
2/ 🙋‍♂️ Hong Kong-headquartered @CryptoCom appointed Tyson Hackwood (@tysonius) to its leadership team as senior vice president and head of global merchant acquisition.

Previously, Tyson led @PaypalHereAU and @Braintree's Asia-Pacific region. Big hire! prnewswire.com/news-releases/…
Read 25 tweets
Profile picture
Felix Mann
@DJFliX
Why is it so hard to have a cross-platform solution to gather link statistics for HomePlug AV devices? I found out that TP-Link uses a platform-native daemon that is called by their Electron app to get vendor specific responses T_T gist.github.com/DJFliX/4bb3a42…
Then there is this guy: github.com/Enrico204/home…
He made an effort to build an application to get data from HomePlug AV devices, but it's dependencies are tied to linux.
But to top it off most of the tools are unmaintained, barely documented, or just plain hacks (for example: github.com/ffainelli/faifa, github.com/qca/open-plc-u…) and of course even if there were this universal tool, the vendors will stuff vendor-specific data in there.
Read 4 tweets
Profile picture
Alec Muffett
@AlecMuffett
Today is a challenge - on and off for the next 12 hours, now that the GDPR dust is settling, I am going to try and Tweet about nothing but the #EUCopyrightDirective - BECAUSE, YE GODS, YOU NEED TO KNOW ABOUT THIS:

techdirt.com/articles/20180…

HT @mmasnick CC @OpenRightsGroup
If you want a video about how YOU should demand that the EU #DeleteArt13 of the #EUCopyrightDirective, maybe start with this one:
If you would like an authoritative voice regards why we YOU need to act to get the EU to #DeleteArt13 - to avoid the Internet and Web being swamped with a "Link Tax", here is the perspective of German Euro-MP, Julia Reda: juliareda.eu/2018/05/censor…
Read 16 tweets
Profile picture
Elliot Alderson
@fs0c131y
When you create a profile in the official @narendramodi #Android app, all your device info (OS, network type, Carrier …) and personal data (email, photo, gender, name, …) are send without your consent to a third-party domain called in.wzrkt.com.
This domain is classified as a phishing link by the company G-Data. This website is hosted by @GoDaddy and the whois info are hidden.
After a quick search, this domain belongs to an American company called @CleverTap. According to their description, “#CleverTap is the next generation app engagement platform. It enables marketers to identify, engage and retain users and provides developers"
Read 7 tweets
Profile picture
Caroline Herry
@_carolineherry
⬇️ Thread de mes sites autour du #design, #inspirations, #graphisme, etc.
N'hésitez pas à ajouter vos sites pref, je vais essayer de les organiser par catégories.
Onglet #Tools > #photos libres de droit et gratuites. Mes préférés sont @PexelsPhotos et @unsplash qui ont le mérite de pas ressembler à des photos de banque d'image lambda et sont souvent très jolies. pexels.com & unsplash.com
En fait, mon problème principal c'est que je bosse rarement sur des projets qui requièrent les superbes photos d'Unsplash... Haha. Comme d'hab, on vérifie quand même la license correspondant à chaque photo.
Read 11 tweets

Trending hashtags

#rstats #ConsolidatedAppropriationsAct2019 #IOHK #ICANN #ThePlan #Justice #IoT #FUD #PIVX #LakeChamplain #Awakening #DoItPOTUS #Java #RealWorldCrypto #jira #trello #PresidentialStatement #8weekfieldprojects #BaaS #gradle #IaC #Creatability #chans #UrbanDictionary #Rs
Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!