Thread by @textfiles

This should be a blog entry but nobody reads blogs and medium is run by fascist-loving techbros and the same fascist-loving techbros run this site so we're trapped so F it, here we are: An Essay

Working with the lovely @archiveteam, when we're not infighting on technique or trying to sort out resource issues, means we spend a lot of time looking out the back hatch watching site after site die. We see it all happening, all the time, and believe me, it's endless.

Many of these sites are just stupid little experiments that got out of hand, or are single-purpose jaunts that our scripts and methods will work just fine on, both capturing the data and putting it up online for "later". Value judgements can come at your leisure.

For example, we've captured every hosted Madden GIFferator creation that was on the site and put it here: archive.org/details/archiv… - if you know the URL of it, you can go back and see any Madden GIF Message people posted. Hooray?

There is, of course, other stuff that's more "important". Where I think Archive Team diverges from a lot of popular opinion is that we think the writings and creations of "just folks" counts as much as anything else, since that's where we see some manner of "what people think".

And yes, there's an entire, smallish group of people who want us to show up at your house with a permission slip to allow us to courier your addition to a website with millions of users into something marked "Not Oblivion" and those folks are VERY LOUD. Let's set that aside

There's another set of websites people seem to think of as "Institutions", which is the closest word I can think of for it. I know people think about them as "Institutions" because they blast me on every social media channel they can find to tell me the Institution Is At Risk.

(By the way, under this definition, 8chan is an institution, but 8chan can go take a backflip into the Goddamned Vacuum Of Space as far as I'm concerned, but we'll set THAT aside too).

Two "Institutions" are closing up in the next week or so. Linux Journal, and Pacific Standard.

Archive Team will grab pretty good copies of these sites, but they won't be perfect, because every modern site is essentially resistant to being copied or used in any reasonable fashion. They'll render in browsers RIGHT THIS SECOND but will almost certainly not, soon.

When you're an "online newspaper", you have none of the attributes of what people probably think of as a "newspaper" in the standard sense. All of your articles are mere funnels meant to throw you at ads or more funnels. Interlinking is rampant and linking externally is weak.

I mean weak in terms of it just breaks over time because you're depending on shifting URLs elsewhere. You don't grab a copy because why would you? You're a newspaper, not like you're CHRONICLING THE PRESENT OR ANYTHING

Additionally, Pacific Standard (for example) renders images on their site with a javascript call, meaning that you offload all the images into this back-end that, wait for it, web crawlers are having a fantastically bad time negotiating. As a result, images generally don't work.

Linux Journal is better than this, generally, but not always. And again, the news articles are not really meant to be "articles" in the strictest sense; they're interwoven code sets within other code sets.

News sites seem to be really important to people, pointing to them as lighthouses in the darkness for printing and being ledgers of truth and occasions to remember. So folks get super-weird when they find out that no, a lot of them are as bad at The Permanent Record as others.

So we did it, we Orwell'd ourselves.

There's lots of folks in the archives space and my opinion is lots of them know this dirty little secret but everyone's very artifact-driven and all the efforts of "archving the web" are uphill from now on, like almost every other online problem (moderation, spam, etc.)

Feel free to go back to whatever it is you do, but be aware that Archive Team is doing what it can against insurmountable, ridiculous odds, and if you don't see value in online content and see the incredible amount of tragedy of its loss, you're not looking hard enough.