Thread by @tleam

#ChaosDay19 talk by Narendra Nalabothula, member of Capital One’s central chaos team. Exploring Chaos in a financial firm.

#chaosday19 : We talk about how failure is inevitable but often couple that with 'service failure is not an option!' This is largely due to the cost of the incident itself. [ed: we often discuss financial impact but should also consider brand/reputation impact too]

#chaosday19 : Chaos Engineering helps us think of failure as an absolute in production. Chaos implementation has a maturity path leading to automation via CI/CD pipeline.

#chaosday19 : We'll be covering various robustness paradigms - immutable infrastructure, bulkheading using circuit breakers, backpressure using reactive streams, and avoiding SPOF

#chaosday19 : Immutable infrastructure provides a number of benefits such as deployment reliability and predictability across non-prod and prod environments. Lightning fast startup speed [ed: critical when dealing with shifting demand] and general improvement to lead times.

#chaosday19 : Moving on to bulkheading - time to help prevent cascading failures. Concept comes from the maritime industry. Theoretically this should have helped prevent the Titanic sinking but bulkheads weren't designed to accommodate the water level.

#chaosday19 : Circuit breakers are our technical equivalent to prevent cascading failure. Service meshes help to provide this capability 'out of the box' which can be useful if you don't want to configure libraries like Hystrix for every microservice in your portfolio.

#chaosday19 : Now getting to how we prevent incidents by implementing Chaos Engineering. Absolutely need to understand the concept of the system's steady state, create a hypothesis. [ed: this harkens back to some of the earlier topics of the importance of observability]

#chaosday19 : Did a quick rush through some of the other needs for Chaos slides. Didn't catch it all :(

/fin

@threadreaderapp unroll please!