Some Reasons The Internet Archive Is So Slow, a quick thread.
With millions of visitors a day, the Archive is one of the most popular websites on the internet - easily in the top 250, occasionally dipping into the top 200. Right now we're #167 worldwide, according to Alexa. alexa.com/siteinfo/archi…
In general, half our traffic is The Wayback Machine, also known as The Thing That Shows You The Old Web, or Online History's Killer App, or Jesus Christ They Have My Geocities Page. It goes by many names. Half of everything hits that sub-site. Everything else is the other half.
Why is the Wayback slow? Well, it's a combination of multiple factors. The saved websites have to be tracked down to the server set that has them, and then the webpage has to be unpacked for you from compressed datasets, and then rendered. It just takes a while.
My boss loves keeping track of what's in there. The Wayback has over 750 billion URLs and related content saved in its stacks, all of them instantly requestable. Instantly requestable does not mean instantly loaded, though.
I watch the Slack channel where Kenji and David and Steve and Mark and Bill and Owen and Corentin and Vangelis and others are all sweating daily to keep the whole endeavor afloat and growing by terabytes and handling hundreds of bugfixes and I can personally guarantee no sleeping 
Another possibility is someone is finding Everything Else slow. Everything Else would be specific items, or specific collections, or doing a search. Let's break those down.
Search is in constant flux for the last year. Thanks for all your hard work, Aaron! Keeping the lights on the extensive search of dozens of petabytes of content going, as well as full-text search, AS WELL as optimizing, adding facet searches, and so on, means it's sometimes slow.
We'll never be as fast as the goog or the bing or the duck but we're also able to do a whole range of searches and deep-scanning that others do not do at the moment, and we're doing it for millions of searches a month, so that's a lot of what's going on. Aaron's on this ALWAYS.
And perhaps, maybe? You're getting some sort of slow download. Like you're downloading a CD image or a 10gb item or a bunch of mp3s and something seems a little slower than you expect. Well, part of that is we don't do tiered memberships - everyone gets the max we can handle.
Occasionally, a blog entry or a huge popular item will mean a set of machines get HAMMERED. We have some "priority" machines we can move a popular item to, and we do a lot of that shuffling, but until it kicks off, the machines might run slow handling the onslaught.
Onslaughts are great, they're what we're here for, and we don't tack on ads or tracking cookies or sell your info or even really keep it while you're doing it. So we don't pay top dollar for maximum everything because there's no shady undercarriage carrying that "awesome" speed.
If there's another reason we're slow or an experience you're finding where things are off, let us know. Let me know. We've found quietly dying disks, a router that's gone flako, an overheated machine, an unexpectedly popular item, and so on, and then we can move in and fix.
The situation we have is that since we DON'T talk to government agencies, ad networks, and privacy-destroying startups as our main (actual) users like a lot of those "fast" sites, we're gonna need to hear from our real users, you. Send along a tip or a note and we'll get on it.
Anyway, that's why we're slow.
