Thread by @copyconstruct

Blogged: Testing in Production, the hard parts

bit.ly/2oa2UYG

Topics include:
- patterns to curtail blast radius
- design of control and data planes
- deployments and lessons from high profile Google and Cloudflare outages born due to a test in prod gone wrong.

There’s a part 2 to this post, which is unwritten as of now.

Titled “testing in production: the fate of state”, it’ll be focused entirely on the testing of *stateful* services in production.

ETA - February 2020. I doubt I’ll have much spare time before January next year.

Thanks, as ever, to the best reviewer in the world @mononcqc. All mistakes and inaccuracies in the post are solely mine.

Thanks also to folks like @lhochstein @aaronblohowiak @hwilson1204 @MarcJBrooker who’ve been very generous in answering my questions over the months!