,
12 tweets,
2 min read
Read on Twitter
Service meshes provide a lot of value, especially around encryption and observability. I question, though, whether circuit breaking/timeouts/retries should be externalized (deferred) to the network. (Thread)
These app-specific (business logic) concerns are impossible to generalize at scale/ in large distributed systems, ie the ones where you typically consider a service mesh.
Some background reading by @tyler_treat bravenewgeek.com/what-you-want-…
It requires a mature engineering organization to develop standards and best practices, as well as complying with them, around the problem of reliability and correctness in distributed communication.
Let‘s assume an enterprise across different engineering organizations and teams buys into service mesh and wants to use features like circuit breaking, retries, etc. Who would write and own the policy/configuration of these service mesh features?
Since devs cannot use (?) code annotations (eg Spring) or property files/structs for related (and proven) in-process libraries, do we assume devs to learn another configuration syntax?
Where do we draw the line? Do devs also own the whole service mesh configuration file? What would the CI/CD process look like? Or do we assume this should be opaque to the developer?
If so, how do we align the communication behavior with the business logic in the code then? (see recent Istio POST idempotency bug raised by @ewolff)
What about systems that don‘t just use RPC style communication, eg async with queues? What’s the net benefit of these aforementioned service mesh features here? (I‘m aware of the WIP - and complexity- on adding Kafka support to Envoy, etc)
To point out, I am not saying service meshes are useless (see initial tweet). I have not implemented a large-scale project in a complex organization. I am throwing out these questions seeking for answers and to reflect my current thinking.
I understand that there a many companies using service mesh (or at least a data plane like Envoy) in production. But I am not so much interested in the success stories of very focussed and stringent eng orgs like Twitter, Lyft or Netflix.
I am looking from a „traditional“ enterprise perspective, with lots of legacy systems, complex and organically grown internal/3rd party apps, diverse engineering skillset, etc.
