Thread by @GeePawHill

UDispatch #3: Custom Traffic Display

Generic traffic display is part of the second layer of UDispatch's intended functionality. And that's okay. But *custom* traffic display is even better.

(Prior articles are:
#1: A Bulk Reverse Proxy
geepawhill.org/udispatch-1-st…

#2: Generic Traffic Display
geepawhill.org/udispatch-2-th…
)

The first layer gives us the handy bulk reverse proxy, and the second one gives us generic traffic display. And both of these help a little. But there's two more layers of function here, and today we talk about third one: custom traffic plugins.

In the early days of implementing upstream-centric architectures, two strong tendencies stand out: 1) an obsession with CRUD as the basic metaphor, and 2) an obsession with limiting endpoints api's.

These tendencies are both right up there with starting land wars in Asia in terms of increasing the likelihood of success. But I'm not here to rail about them. Rather, I want to point out one significant consequence of them: the size and complexity of the response content.

I have seen and worked with upstreams that return JSON to the tune of 25K characters per call. And here's the thing: for the purposes of the particular downstream I'm writing, 98% of that return was useless and hard-to-read noise that obscured the signal I was after.

(Do you see how this is a direct result of both CRUD-obsession and endpoint-as-scarce-resource? It means whole entire backend objects containing everything known to the service are flying around for even trivial calls, like "change the release date".)

And here you are, you've got this lovely rendering of the tree-shaped data from one of these responses in the generic JSON render, and you can tell you got good data by just scrolling 85 times until you can see the two fields out of 1000 that your operating changed.

Dangit. If only you had an easy way to do a *custom* render in UDispatch instead of that generic whole-tree render.

Enter the third level of UDispatch: rendering plugins for custom traffic.

The basic concept: When UDispatch runs, it loads the rendering plugins, which basically consist of two parts, a recognizer and a renderer.

As every message comes in, UDispatch calls the plugin's recognizer (with the content and a couple of extra fields that are useful for recognition). If it's recognized, UDispatch gives it a tab to render in.

The advanced concept: UDispatch comes with the ability to point at different remote plugin repos. It can easily install them on the spot at the user's request, even when they come in a set.

The *tricksy* concept: UDispatch has to be pretty smart about versioning. The plugin for rendering build 1920 might need to be very different rendering build 1921. And you might need them both alive at the same time. And you might have the *upstream* team updating them.

So this is the third layer. Its point is just to trim a little more time out of the making.

The 4th, maybe 5th, biggest myth developers accept more or less as writ: that "human-readable" = "human-efficiently-usable". I've worked with XML files measured in gigabytes. Even setting aside that most of it was duplicate text, most of the *rest* of it was opaque to my app.

(Reminder: "opaque" in this context means "irrelevant to my application". Sometimes that means "entirely unused anywhere", sometimes that means "passed on to some later victim, not my problem".)

So if what matters to my downstream app is three fields describing the order status, *that's* what the custom render shows, not the 193 other fields.

Now, the *deep* fix for that is to change one's life: multiply endpoints like bunnies, and stop writing CRUD endpoints. But as far as I can make out, Denzel Washington is still the emperor, and until I am promoted, the practice -- the *law*, the *enforcement* -- won't change.

Of course, custom rendering can do more than just narrow and focus a big tree. You can use it to screen-scrape the important parts of an HTML response. You can just make a tree look like a nice clean form.

Some services, hopefully your new ones, aren't mostly worth the effort of custom rendering. In others, a couple of custom renders can save massive amounts of time during the making.

The third layer adds the means to make UDispatch plugins that render its traffic in ways that are very specific to your downstream, and, depending on your rendering need, dead easy to write.

Soon, soon, we'll get to the fourth layer. A little foreshadowing here: "Hmmm, what *else* could you do easily with plugins?"

MuhuhahaHAHAHAAAAAA!

Have a great and comically evil Friday!