Thread by @rmoff

#Current22 @AdiPolak talking about chaos engineering

A scary list of all the things that could go wrong with data flows #Current22

“Disagree and commit” h/t @matryer #Current22

What can we learn from the software world of chaos engineering and apply it to the world of data flows?

Principles of chaos engineering

#Current22

Comparing steady state meaning in devops/SRE world to that in data #Current22

“The data isn’t wrong; your expectation of the data is wrong”

Chaos engineering - varying real world events. In data context this could be schematic changes, data corruption, fubar with partition deletion…

#Current22

No one wants to work weekends…

#Current22

Testing in production. Which in the data world means using production data 😱 #Current22

The stages of a data product #Current22

“git for data” with @lakeFS

#Current22

@lakeFS is an open source project, written in Go. It uses copy-on-write to efficiently provide duplicate copies of files. #Current22

Using quality check hooks to protect production data

Live demo time at #Current22

Creating a branch of data

“High performance yarn” is “fiction” 😆 #Current22

What if the join doesn’t work as we intended it?

Uh oh, we’ve got nulls

Now what do we do? Throw away the null data? Try and replace the values? How about just rolling back to before we made the change. #Current22

Now we fix the join and do it properly. We do the same data checks again to confirm it.