(The reason I go through comments on HN/etc. is that for every 6 "the author is a platypus" and 3 "amen" there's 1 person genuinely thinking about this for the first time and it's a worthwhile exchange)
Kudos to all the geniuses who think my get rich quick scheme is to argue a nuanced viewpoint every couple years and being on the receiving end of a bunch of hate mail and a bill for the bandwidth.
Y'all got me all figured out.
I love how the Go team can say "we literally don't care about language design, anyone who does is a nerd and we don't hire nerds" every occasion they get for eight years straight and when I say "they didn't design a language" suddenly I'm the asshole
see, it's not just me.
unflagged after 2 hours. the cycle is complete
flagged again, removed from frontpage. nothing to see here, carry on!
reasoning for taking the article off the front page:
I don't know, maybe if your community rips itself apart two days in a row reading articles /I did not post there/, the problem isn't the articles.
This isn't censorship, they can do whatever they want on their site but it did seem like a LOT of people wanted to read/discuss this
see, this is what we have content warnings for:
the rules of this city are very unclear
fasterthanli.me is sustaining a DDoS attack: 6M requests in the past 2 hours, mostly from China and the US (they're just hammering the front page).
I had to turn on Cloudflare's attack mode, sorry for the javascript challenge.
Although most of it is static /content/, my site works more like a CMS: there's DB queries (sqlite) involved, templating, HTML rewriting, etc - here's what the server is spending time on.
I never needed to add caching (despite HN) until today 😊
There's about 40K concurrent connections right now fighting for access to a single SQLite database, it's time to set up some rate limiting too.
250K current connections now - server process is only using 800MB of RAM, which is nice. Setting up rate limiting soon, but I'll have to limit connections in some way as well. Luckily I've written that kind of code often before 😊
Oh, would you look at that, it's back up and fast again.
oh it was very complex, I'm gonna have to use a picture for this
(joke aside, cloudflare is dealing with most of the nonsense here)
oh, cloudflare does not cache static HTML content by default hahaha was that always like that? guess they have to get people to buy page rules
well, the attack stopped for now. if it picks up again I'll have to implement actual caching on my side, since cloudflare doesn't let me do what I want (cache HTML, only for logged-out users). but for now I have other stuff to do.
The attack came from Singapore, Los Angeles, London, Hong Kong, Frankfurt. Using 4 Chrome-like user agents. Mostly resulted in 503/403/499/524.
More details on the the attack & how cloudflare reacted to it (from their Security graphs)
Welcome back DDoS kids! I'm ready to take care of that next wave, let's see how this go.
To whoever is behind the attack: thanks for the free load testing service (doing it myself is against the ToS of several things), this is costing you more than it's costing me and I'm getting entertainment + experience hardening my stuff against it. A++
Hey no come back! the attack stopped too soon, I was just about to deploy a change :(
Just to show you what I'm seeing:
attack: 350K connections
no attack: 5K connections
Ah yes I forgot cloudflare doesn't know how to close connections so a 128 limit means the site is unavailable after just a little while. I need me some idle timeouts.
Love to pay for slowloris as a service.
attack is pretty well distributed, I'm not gonna block a whole country over it though
the logical next step is adding some observability to this codebase (and then some actual caching) but I'm gonna chill for a bit and maybe do that later
the problem if I add observability as-is is that I'm gonna blow through the @honeycombio plan limits
unless somehow those limits were lifted for the next 48hrs or so (🙏) I'd be blind until the next billing period so that's not good. (but today's the 30th? idk)
note: this is ABSOLUTELY NOT the "go team" or the "go community" reacting to my article. this is one person, a small discord, or some channers having their fun.
the go team got sad and blocked me, they'd never do a DDoS lol
front page of HN, day 3 😬
big bookclub energy
in all that commotion I hadn't noticed my video platform was attacked too
the attacker left me a lil' message in the user-agent
oh duh, my SQL code 1) doesn't cache prepared statements, 2) runs blocking code in an async context (should use spawn_blocking)
I missed a bunch of low-hanging fruits in the initial implementation, this'll be easy to fix.
also telemetry is all set now - gotta cache that feed!
this is why I love observability: I'm not happy about these SQL queries but I had no idea I was spending so much time truncating HTML.
there's a bunch of trivial ways to fix this, which is great news!
I thought my load testing tool broke, but no, it just completes instantly now