,
13 tweets,
7 min read
Read on Twitter
#chaosday19 @krisnova : Managing chaos - how do we prep for something we can't prepare for? How do we get ready? Normally we manage it by building an "awesome" script to fix it all.
#chaosday19 @krisnova : How do we get to that script? Well something goes wrong. Time goes by. Humans get involved to repair, we triage, and then figure out the fix. Then this turns into a shell script after we go through the same process multiple times.
#chaosday19 @krisnova : What makes Kube so successful? (audience: Marketing! Cloud lock-in! *cue laughs*). Most important thing: declarative nature.
#chaosday19 @krisnova : Control loops - consistent evaluation of current state, determining next desired state and then taking action. Repeat ad infinitum. [ed: Think OODA loops. en.wikipedia.org/wiki/OODA_loop]
#chaosday19 @krisnova : Reconciling a simple VM simply 'restarting' elements of a node isn't enough. If you just replace 50GB of disk, what happens to it's content?
#chaosday19 @krisnova : What could possibly go wrong that you might not be prepared for? Over time we reduce the number of unknown-unknowns in our control loop to reconcile issues.
#chaosday19 @krisnova : Getting into Kube CRDs (basically whatever you want it to be - literally key/value pairs) and Operators (things that identify when things go wrong). Or... simply they are literally YAML and a while() loop.
#chaosday19 @krisnova : the combo of CRDs and Operators help us continue and automate the initial kornshell loop mentioned earlier!
/fin
/fin
#chaosday19 @krisnova : Q&A - what about when an operator acts differently than expected? This is where status comes into play so you can identify what was going on.
#chaosday19 @krisnova : Q&A - An update to the YAML seems like a global config push which is risky how do we mitigate? Some tools in place like GitOps and then using an operator to reconcile.
#chaosday19 @krisnova : Q&A - How do we deal with scenario where we declare something but it wasn't respected? Depends on the maturity of the operator. What does your failure look like? Maybe the program should send an e-mail. This gets into opinions re: operator purpose
@threadreaderapp unroll please!
