Thread by @_tr

I did some reading on @EnvoyProx's locality, and it is SO much more powerful than I had previously thought. It's crucial to implementing a service mesh with dynamic config, and it even has implications for how load balancing works.

Let's have a thread!

First, wtf is locality?

Locality is just a struct that Envoy defines about itself. Each proxy instance can be configured to know about which zone it lives in (e.g. datacenter), which cluster it typically serves (e.g. ServiceFoo), and some other opaque metadata (e.g. AZ-VA-12).

Envoy can be dynamically configured. There are a set of APIs—collectively, "xDS", for "x Discovery Service", where x is Cluster or Route or w/e. When an individual Envoy asks for its config, it sends its locality to the management server.

First realization: the locality is crucial if you're trying to create a service mesh. Proxies that live next to ServiceFoo should forward their requests to ServiceFoo their local service. Other requests should get handled elsewhere.

This isn't a problem in previous worlds, proxies weren't configured dynamically. But in Envoy's world, you can run it both for intra-service traffic and as the primary load balancer for all traffic ("front proxy"). You need to know the difference!

OK, so whatever, that's just adding complexity.

The cool thing is, by setting these identifiers, Envoy can actually make intelligent decisions about how to route traffic between zones. Intelligent = try to keep in local. Intra-zone traffic is slow and costly!

THIS IS BUILT IN. For instance, if you let Envoy do all the work, it'll try to keep traffic from Zone A going to instances of Service Foo in Zone A.

Only when there are failures in Zone A does it spill over traffic to Zone B. If you have a little failure, it does nothing. If you have a lot, it'll send enough to keep the overall load on the services balanced. Very cool!

If you want control of this load balancing, you can have your management server just return the weights directly. The best part? Envoy will still do that gradually-fail-if-this-zone-is-unhealthy thing, even when you give it target weights!

If you want full control, Envoy also has subset load balancing. In this mode, you can just write rules that make any sort of routing decision based on any characteristic about the request.

But here's the deal: all of this works together. AND ONLY TOGETHER. If you want to be multi-region, you need localities. If you want a service mesh, you need dynamic config. And you can mash these things up in your management server!

Second realization: once you have all that control in one place, a TON of complexity comes out of your service code. Services can just ask for another service, and while can customize how they get the right instance, it's independent of what service is asking.

It could be in a different zone.
It could be on different infrastructure.
It could be different depending for employees vs. customers.
It could be different if you're in incident.
It could be different if you're mid-release.
It could be different on Tuesday.

IT IS ALL UP TO YOU!

Basically, if you have a more complex topology than a single AWS AZ, and more than a single service, you'll want to take a look at Locality and see how it can help. Everybody is different, and Envoy has good defaults, but there's a TON of power in here if you customize for you.

Go do it! Do an Envoy thing!

And read learnenvoy.io for more of the good stuff!