Silvio puts a finger on something I’ve been thinking about for awhile, but not articulated before: the slow, long-term shift in day-to-day programming practice from algorithm-centric to API-centric.
Some thoughts on what this might mean for us (1/?)
Some thoughts on what this might mean for us (1/?)
It used to be that day-to-day, being a programmer meant spending most of your time writing code for a system that all fit on your box. Even if it was a network app, like a website, you could start the whole thing up locally to run tests, either automatic or manual.
Maybe you made a network call or two out to third-party APIs. For testing, you could either stub those calls out (i.e., record a real response once, then replay it for tests) or use a sandbox endpoint (if the provider had one, or it was important enough for you to construct one).
Over time that proportion has shifted. Rather than the occasional API call surrounded by lots of local processing, more of the app is orchestrating a series of API calls across multiple providers.
A large library of stubbed responses is a pain in the ass to maintain - especially if they depend on each other, are designed poorly (i.e. require substitutions to be realistic enough), and/or are at different levels of stability.
Some shops double down on local control, and write elaborate scripts to spin up all necessary services locally (or sets of services per-developer in the cloud). But in large orgs that are all-in on services, that could mean hundreds of individual processes.
At some point during this transition, you will find yourself operating as part of a distributed system that is so complex, with so many moving parts, that you cannot, necessarily, test it under realistic conditions.
I feel like the conceptual shift in your architecture this requires has been well-covered - testing in production, going from monitoring to observability, moving to an async model where everything can be rolled back later.
But what I haven’t seen well-covered is how we should shift day-to-day operations at the individual developer level.
Because all our IDEs and build apparatus and testing tools are holdovers from a world where algorithms weren’t so distributed.
Because all our IDEs and build apparatus and testing tools are holdovers from a world where algorithms weren’t so distributed.
As @betsythemuffin puts it, the “illusion of local control” can no longer be maintained. It was, of course, always an illusion, but one that our tools made it possible to mostly believe.
A few what-ifs:
What if instead of being oriented around codebases, your IDE was oriented around a concepts that could span multiple codebases?
(Yes, a concept “should” be contained in one...but we all know the reality.)
What if instead of being oriented around codebases, your IDE was oriented around a concepts that could span multiple codebases?
(Yes, a concept “should” be contained in one...but we all know the reality.)
What if your editor and build tools didn’t assume that you had a primary development language?
What if your editor knew about the APIs you interacted most closely with, and “indexed” them, like today they index libraries?
What if you could easily generate a stub service for your close collaborators to use? What if your system could easily find runnable stubs representing prod/staging/dev of your close collaborators?
What if versioning your APIs (and maintaining old versions indefinitely) wasn’t a huge deal? What if it wasn’t a huge deal to pin to a version of collaborator, because you knew it would always be there?
It took us decades to properly solve the local package management resolution problem, even within single-language ecosystems. Now we’ve taken that problem and exploded it across ecosystems AND the network. It’s really not clear what a good solution even looks like here.
Or even if we’ve got the problem right. Your thoughts?
I know I’m too stuck on control - most of my examples are “what if we pretend we still know what’s going on elsewhere.” So I’m particularly interested in your takes on what it looks like to embrace the not-knowing.
Right now though I’m going to go to the beach & have lunch 🍔
Right now though I’m going to go to the beach & have lunch 🍔
