Profile picture
Patrick McKenzie @patio11
, 10 tweets, 2 min read Read on Twitter
A long time ago, I worked for a Japanese company which had a Alert Lots Of People function available to it. We drilled on its use.

Since it's instructive (this thread may be longish):
A: "I am declaring [an emergency / a drill] for [area]. Get the book."
B: "I am secondary on [an emergency / a drill] for [area]. Here is the book."

A flips to Page 1 and reads down a checklist in order.
A: "I am attempting to log into [production / test] environment."
B: "Confirming you are attempting to log into [production / test environment]."

A: "Login complete."
B: "I verify you are logged into production. Proceed."

A: "I am queueing up an alert, entitled $TEXT."
B: "Confirm to me that your expected text is [reads screen]."
A: "Confirm"
A: "I am sending this alert to $GROUP."
B: "I acknowledge you will send the alert to $GROUP."
A: "I have selected $GROUP in the interface. Confirm."
B: "Confirmed."

A: "I will now send the alert."
B: "You are good to send the alert."
A: "I have sent the alert."
They then verify that the alert was in fact sent and verify it was received on one of the canaries, using a similar level of pedanticness.
This could be circumvented by any engineer, on their own authority, if there was a danger to human life. (e.g. "I became aware of a tsunami at 3 AM in the morning and could not raise a second within 15 seconds.")

We thought about this process *a lot*, and drilled it.
You can file this under "an anecdote from an engineering culture which does a few things really, really well"; this is one of them.
"Was it an actual, physical book?"

Well, technically, a binder, in case we ever needed to update procedures.

"Why not have it on an electronic system?"

Because binders have 100% uptime even if office/remote servers/etc are presently rattling in the wake of e.g. earthquake.
"What happens if the Internet connection was down between you and the system?"

This is a question that gets perilously close to that I Can't Tell You territory, but web developers who have ever coded "Fire X if we don't get a heartbeat from Y" can probably guess.
The above is offered in the spirit of improving plans at your current/future workplaces and not as even an implied criticism of any other systems.

When it really counts, there are only two teams: humanity and disaster. We’re all in it together on Team Humanity.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Patrick McKenzie
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert is as a PDF, save and print for later use!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!