Thread by @QuinnyPig

This cloud thing is a fad. I'm building my next thing in a datacenter instead.

I fly to a city to tour DCs, because I'm a responsible grown-up.

The first one is staffed by super friendly folks. They wave me in. I'm a prospective customer, "they don't need to see my ID." Cross them off the list.

The second one shows me their electrical (or "one-line") diagram. It shows four transformers. We tour the site. There are two transformers and two concrete pads. NEXT.

The third one seems fine. We shake hands. I can move in starting Q2. I sign a contract committing me to pay them for three years.

I order servers and have them drop-shipped to the datacenter. I'm careful to schedule arrival so I don't have to pay storage charges. Dell fucks it up.

I order networking gear and have it shipped out. I grab the wrong part number and get a GBIC I don't need. I file a return. Cisco fucks it up.

The cage build-out will take me two days. I book four days because this isn't my first rodeo and I know how this works.

I order spare hard drives to leave in the cage for when my hardware inevitably dies.

I remember to stock a full tool-kit that will remain on-site so I don't have to remember to bring parts I need at 2AM.

I put the rails in the racks. My integrator cheaped out; I need to use screws instead of quick snap rails.

The cage nuts and the screws look like the same size; they are not. Fry's fucked it up.

Build-out day arrives. The PSUs are for 120v; the cage is wired for three-phase. I begin swearing.

I rack eight identical servers. Seven work. I begin diagnosing.

I planned ahead and have a cable tester, so I only lose two hours finding the bad cable.

The cross-connect is--fiber?! I need that GBIC after all. I reread the networking order. Of course CenturyLink fucks it up.

I'm too impatient to enable Spanning Tree. I inadvertently create a bridging loop. I bodily rip the cable out of the switch. TCP now terminates on the floor.

I call it a day and go to the hotel. At 2AM the pager goes off as my site starts dying. I rush back to the datacenter. A "helpful" NOC employee has been racheting down the "loose" fiber runs. We exchange words and almost blows.

One of the switches has issues and I can't configure it. Everything supports MDI/MDIX autosense except for networking gear. I make a crossover cable while swearing.

I rack a ninth server. It doesn't work. It turns out my assistant "helpfully" put the bad cable into the spares bin. I refrain from whipping them with the cable and cut it this time.

I configure screen on my Macbook to talk to the serial-to-usb adapter I plug into my borrowed rollover cable.

Things are finally working. Should that stop I can either come back, or pay $75 an hour for remote hands to poke around on my servers for me.

I fly home and begin writing my app. I'm sure glad I'm not locked in to a cloud provider!

(Nothing in this thread was fictional. Every single thing here happened to me in my datacenter days.)

"Hey I just woke you
And this is crazy...
But your DC's on fire.
Fix it maybe?" --@pagerduty, 3AM

What, you thought it would be that easy to end this thread?

A machine isn't speaking to the network. DRAC is non-responsive, so I pay remote hands to power cycle the box.

Remote hands power cycles the wrong box despite me labeling the servers clearly and correctly.