Thread by @GeePawHill

In our efforts to optimize the Many More Much Smaller Steps (MMMSS) path, we've tried and rejected the "shortest-distance" floptimization. Today, let's take up the "pin-making" floptimization, in which we create specialists, stations, and hand-offs.

geepawhill.org/2021/09/29/man…

Your periodic reminder: as much as I enjoy thinking and writing and talking about changing code, changing the world is far more important to me. Please join me in supporting and enacting change for social equity.

Black Lives Matter.

The seeming awkwardness of the MMMSS path pushes us to propose various optimizations. Unfortunately, most of these are based in analogies that flop, they don't actually improve that path. We call them "floptimizations", and today we'll take up the pin-making floptimization.

In this floptimization, we notice three things: that there are separable activities, each of which takes an input and makes from it a different output, that the activities benefit from specialization, that the activities can be seen as "stations" in a process flow.

We take these activities, and we arrange them into a directed graph, at its simplest, a straight line from one activity to another. We assign individual workers to each activity, and each takes an artifact from the previous activity and hands it to the next.

Why "pin-making"? Because one of the earliest strong formulations of this idea comes from Adam Smith, the Scottish economist who's regarded as the founder of capitalism, and is in the form of a description of a factory for making pins.

(Irrelevant historical aside: No fair hating on Adam Smith, who was a great thinker and a profoundly moral and decent person. If he saw extractive American-style hyper-capitalism as it is today, he'd most likely be horrified by it.)

In Smith's pin factory, "One man draws out the wire, another straightens it, a third cuts it, a fourth points, a fifth grinds it at the top for receiving the head," and the head itself is made with several distinct operations before being attached.

He identifies up to 18 distinct "stations" in this process, and describes a working factory in which some of the stations are run by the same individual, and ten people can make 12 pounds of pins a day.

In the "pin-making" concept, then, we have division of labor and we have artifacts moving in sequence through a series of stations run by specialists. For many of my readers, this should sound familiar already, but eventually we'll make it even more so.

A pin-making arrangement greatly resembles a kind of machine, only with some of the parts being humans. (That's a problem, of course, but not one we'll address today, we'll get there, trust me. :) )

Pins, today, are made not by humans but by rather lovely intricate machines.

The theory behind the pin-making idea is that, by breaking a step into actions, getting specialists to perform each action, and moving the artifact between these specialist stations in a steady flow, we will finish that step more quickly.

The stride will be shorter.

Seems reasonable. So what makes us call it a floptimization? It flops because a shorter step that's longer than our optimal stride length doesn't help much. And it flops because it doesn't actually shorten steps in most high-collaboration high-creativity settings.

Imagine we have a step that is a little longer than our optimal stride-length. It's possible, in theory, that pin-making will shorten that step, such that it would actually fit in our stride. If it does, then certainly pin-making would be a good idea.

But before we consider whether it does or not, notice the very tight constraint of that condition: we have a step that is "a little longer than optimal". That's a narrow applicability. Of all possible steps, the overwhelming majority of them are *much* longer than optimal.

And when we apply pin-making to a too-long step, but it doesn't make the step short enough to be inside our optimal stride-length, well, we're right back in shortest-distance thinking. We just spent a whole long article explaining why that doesn't work.

So then, what if we only apply the pin-making strategy when a step is of "the right size", a little longer than optimal?

Oh, you sly dog!

As so often, in theory that will work, because theory doesn't charge us for preparation or control or communication.

Smith's pin factory had a dozen stations and specialists. Those stations were physically prepared in advance, and those specialists were hired and trained for their specialty, both activities of some cost, even in pin-making.

To optionally apply pin-making on an ad hoc basis, we'd have to have our non-pin-making arrangement, our pin-making arrangement, our stations and specialists, and switch between them based on our ongoing and continuous analysis of the situation.

Yikes. No way that's free.

So the point of this first objection is a) that using pin-making to shorten very-long steps into merely too-long steps suffers from shortest-distance problems, and b) that we can't generally just apply pin-making in an ad hoc fashion.

Notice, tho, the first critique assumes pin-making actually works to shorten most steps. "Regardless of step-length, pin-making will take nearly any step, make it more efficient, and effectively shorten its duration."

Is that actually even true? Welcome to the second objection.

There are three factors that make Smith's pin-making a reliable optimization. 1) The artifact is all that has to move between stations. 2) The specialists reliably and significantly out-perform non-specialists. 3) The production system is a directed *acyclic* graph.

Well. Whoops. In fact, in the software development trade, none of these three factors actually holds. Let's take a look.

First, in the pin-factory, the only transfer from station to station is the singular artifact itself: A wire, a straightened wire, a sharpened wire, a pinhead, and so on.

In that setting, nearly all of the communication and collaboration is vested in the singular concrete physical object moving between stations. The moving artifact is both necessary and sufficient for the optimization to work.

The artifacts in the software world -- source text, binary images, test results, wireframes, screenshots, flow diagrams, bug reports, and so on, are multiple in nature, partial, abstract, and complicated. They may or may not be necessary, but they're definitely not sufficient.

Examples: "Use the usual sidebar, I didn't show it." "It's ready, but the frobisher can only handle blitbanks, don't use flaprods." "It's on the shared drive, something-something-Controller-Design." "It happens when you use a name containing Z on UK time on Thursdays, sometimes."

Every one of these litle snippets, and I'm sure my readers have seen several others just today, is an example where *more* than the artifact is necessary. It's an example, in fact, of the high need for active collaboration in software development.

The artifacts in software are just not as simple and orderly and repetitive as a box or frame holding intermediate pin parts. It's not possible to move them from station to station without a very great deal of both support and communication.

Second, in the pin-factory, our specialists reliably and significantly out-perform our non-specialists. And to elaborate on this, they do the same thing over and over, they do it day in and day out, approximately forever, and they do it harnessed but in solo activity.

There are a lot of specialties in the software trade: front-end, back-end, language-specific, database, UX, product, testing, documentation, and so on. And, given advanced solo tasks in their specialty, they do, often, out-perform non-specialists.

So where's the problem? Well, it's several small problems. We won't go into them in detail, but let me throw a napkin sketch at you, instead.

1) Most tasks in the trade are *not* advanced, so we gain less benefit. 2) It is rarely possible for a specialist to work in solo mode, see above. 3) Our specialists aren't doing the same thing over and over, but different things in the same domain.

4) Specialists are expensive, so we have to optimize their load. 5) Specialists are de facto knowledge and technique and language silos. 6) Specialists aren't machine parts, and they get bored and de-motivated. 7) Specialists are bottlenecks just waiting to block flow.

Now, don't take this wrong, We're not in any way suggesting that we can't benefit from having a highly diverse range of talent and experience and interest in our teams. We *need* people with special skills and interests.

We're saying that turning these people into single-purpose stations, giving them a job that is always and only one kind of activity, is a grave error in software development, which is a creative and collaborative endeavor, not an assembly line.

This is not a factory.

And the third factor in Smith's pin factory: there are no cycles between stations. We don't move from wire-straightening to wire-pointing and then back to wire-straightening. There are no loops, and though there are occasional back-steps, they're uncommon in manufacturing.

In softare development, of course, cycles are absolutely an everyday thing. The most common ones occur between development and some kind of validation. Cycles have been built into software-development pin-making systems since the earliest days.

(We often say "waterfall" when we mean "no cycles". This, as it happens, is completely untrue of Royce's waterfall: his paper & idea has cycles built into every phase. Vilifying a complete mis-representation of someone's ideas is as old as humanity.)

The cycles we notice most often are the ones between a developer and some specialist validator. But very often, there's a ripple-effect, where resolving one validator involves not just the developer, but multiple other validator-specialists as well.

A single example will suffice: the developer "finishes" the appointment-booking page. UX says, "don't sort months in the combo-box alphabetically". Product says, "don't let them book more than 3 months in advance." QA says, "don't allow people to book events in the past" ...

And each one of these sends the source-code artifact backwards in the flow, back to the developer "station". Further, because the booker is complicated, each trip back to the developer invalidates previous testing and requires those stations to re-process it.

Further, at larger scale, the nature of the modern distribution system is such that the whole *project* proceeds in a series of cycles, from inception to market, then market to enhancement to market to enhancement. Each enhancment puts the same artifact back in the flow.

So, what these three missing factors -- insufficient artifacts, comparatively low value from station-specialists, and cycles in the flow -- what they all mean is that the conditions under which pin-making would normally be an optimization simply do not hold in the geek trades.

To summarize the whole argument, then, there's just two ideas: 1) If pin-making worked in software, it wouldn't work often enough or beneficially enough to justify its usage. 2) Pin-making mostly doesn't work in software.

Just like the shortest-distance floptimization, this one sounds good, and it looks really cool on a Jira state diagram. But, in practice, pin-making for software development is just another flop.

And there's one more aspect to all this: drawing up a chart of stations and handoffs and artifacts, hiring to fill them, adding parallel streams to scale, load-balancing to optimize, all of these activities are in fact stunningly difficult to do.

In fact, these are all so incredibly difficult, it would take a *genius* to be able to pull it off at all. And that, my friends, is the third idea we'll be considering in the coming days: the mastermind floptimization, knowing all, seeing all, controlling all.

Seeya soon!

Thanks for reading along. If you like my work, here's how you can help:

1) Subscribe. Free, spam-free, full-text or audio. geepawhill.org/subscribe

2) Keep working for and supporting change, in the code, the team, the organization, and above all, the world.

Black Lives Matter.