Okay! Many folks have pinged me for the ground truth on the UniSuper thing and I now better understand exactly what went down, and WHEW am I relieved.
Punchline: this failure mode cannot affect other @googlecloud users.
Lemme explain:
Punchline: this failure mode cannot affect other @googlecloud users.
Lemme explain:
First More Punchline: The trigger for this incident only affected resources *inside* one service with a concept called “private cloud” that is just for that service - not their Google VPC or the broader Google account or org or project context level resources.
Google teams detected and dove on the problem in minutes and worked hard together with the customer towards resolution. Way to go homies. #hugops.
So what happened??
Moral of the story: check your args. Lemme break it down:
So what happened??
Moral of the story: check your args. Lemme break it down:
A manual script used for this deployment passed a null value where it shouldn’t have, a value that had the unintended effect of creating a one-year subscription rather than a normal perpetual deployment.
I bet you can tell where this is headed…
I bet you can tell where this is headed…
You guessed it, a year passed, and Google’s tooling did what the time-limitation required and initiated turndown of this Subscription (there’s that magic word) and this specific Private Cloud instance and all resources within it. Eeeek.
Why can’t it happen again? Thankfully this method of deployment has been deprecated for months, so it cannot affect others. Adios bugs!
Everyone on that one service is no longer exposed to this risk, and all other types of Google Cloud services (the vast vast majority) were never exposed. No rewriting the backup policy manuals, Google Accounts not implicated, again, vast WHEW.
Happy to answer any Q’s or help folks noodle this thru.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
