Thread by @GeePawHill

You can put all your configuration in a shared library and eliminate just about every mis-configuration in your multi-process application. It's not free, but it's cheap, and it kills a lot of minor pain. Let's take a gander.

It's Sunday geek comfort-food time. I hope you enjoy it, and I also hope you remember it's not the most important story out there. Please keep working for change, once you're rested.

Stay safe, stay strong, stay kind, stay angry.

Black Lives Matter.

A multi-process app is one where there are several programs that are built and provisioned separately, but must collaborate to create the experience of the user. All micro-service architectures are multi-process, for instance, but there are other apps like that, too.

In order to collaborate at runtime, these programs need to share things like endpoints, URLs, feature flags, and sometimes even file paths. Mis-configurations can cause lots of wasted time, in development, testing and release.

By far the most common way to achieve this sharing is by maintaining some variety of "configuration profile" files, one for each program. These contain ASCII -- YAML or environment-variable style -- depending on and customized for the particular program.

When we have two programs and two profiles and the profiles don't agree about one or more of the configuration settings, the result is usually, tho by no means always, a kind of easy "wait, what?" problem.

What I mean is, the programs tend to collaborate incorrectly in such an obvious way that we usually notice it pretty quickly. We go double-check the configurations, spot the incorrect ASCII string, hand-edit it, re-up it, bounce it, and hopefully we're off and running again.

Of course, they're not *always* that obvious. I've seen multi-service apps mis-connected for days at a time, causing confusing results, intermittents, flickery tests, 'sploding demo sessions. 95% are easy, but 4% are hard, an 1% are "Sev 1 all hands on deck".

But they are *usually* cheap, some of the easiest bugs & fixes we encounter. The problem, of course, is that they happen just often enough that they slow us, but not often enough that we take a hard look at what we're doing.

What *are* we doing? We're violating three different pieces of advice: "Don't Repeat Yourself (DRY)", "Reduce Mental Bandwidth", and "Easiest Nearest Owwie First (ENOF)". We're doing a hat-trick of bad habits here.

"Don't Repeat Yourself" is a heuristic that tells us to avoid duplication, because if there are two places to keep the same code or data, we will forever after be required to keep them in sync.

"Reduce Mental Bandwidth" is telling us that the most common answer to these problems, which is "everyone remember to always change or check both of these profiles at the same time", is putting a burden on the human brain that it can't reliably bear.

"Easiest Nearest Owwie First" is saying that, even though there are bigger problems than misconfiguration, we still want to fix misconfiguration, because it's within our reach, it's easy, and it's an owwie.

There are several possible ways to kill off misconfigurations like these. But the one I like the most is one people seem to implement the least: use a shared resource to make config simpler to do, simpler to control, simpler to version, and simpler to test. Here's how it works.

Create a repo with code contents that are one way (or more, see below) to make filled-out non-primitive Configuration objects based on simple function calls or even just keywords. Write your project to use those instead of ASCII config profiles.

Detail: The configuration-making is purpose-built to supply only legitimate configurations. In most environments, there are fewer than a dozen of these, tho configuration-by-string lets you create thousands or millions of broken ones.

Detail: We said "one way (or more)". All of our programs aren't written in the same language. No worry, your repo can have one in the backend language, one in the frontend, one for that weird old thing written in caveman days, one for those cutting-edge bozos over there.

Detail: Why non-primitive? This is a very long conversation, and we're just going to have to work on it another time. The too-long-didn't-write: so we can give it custom behavior when we want that, which we will.

There are several real benefits to this kind of approach -- honestly, to *most* of the competitor ideas, anything that isn't "same strings in many files" -- so let's list a few of them.

1) It's cheap. Adding a repo, writing a first pass at a Configuration, these are not rocket-science designs, but very basic stuff, especially in the beginning.

2) It's testable. Since I have all and only the legal Configurations for all my apps, I can roll tests around all of the decision-making process and its resulting values, long before a client app ever gets deployed.

3) It's iterable and incremental. Programs can be stepped over to using this one app or even one value at a time. This is a huge benefit, because we get value not once, at the end, but distributed, every time we take a step.

4) It makes the implicit explicit, nearly always a winning move in geekery. Config-by-profile has lots of implicit rules. We can put those rules directly into our Configuration scheme, express them in code, and prove that they aren't violated.

Two of my recurring themes are triggered by all this: 1) It's this way because we built it this way. 2) The code works for us, we don't work for the code. This is a really straightforward case, the problem from theme 1, the solution from theme 2.

We *chose* untested by-hand ASCII configuration profiles. We didn't have to, but we did. With a scheme like this one, or any of the other ideas with similar values, we are making the code work for us.

So think it over! I can't give you up-close details cuz I'm not in your codebase. But I can tell you, it's actually pretty easy. The hard part of this, as so often, isn't implementing an answer, it's looking for it in the first place.

A shared library for configuration is a way to kill off a bunch of small, annoying, persistent problems, in development, in testing, and in production. It's not free, but it's cheap, and it pays off well and rapidly. Queue up a spike and give it a try!

Thanks for reading along, a short one today. Subcribe! It's free, spam-free, and you get full-text or audio.

geepawhill.org/subscribe

And folks, please embrace change *outside* the monitor. We can fix this, we're the only thing that can.

Black Lives Matter.