Amazing and sobering WSJ interview of Dutch Admiral Bauer, NATO's top military officer, on supply chain risks.
"We've long thought that mutual economic relationships between nations would prevent war. We thought we were buying our gas from a company. Turns out that wasn't true.
"Putin told Gazprom to turn the [gas] off, despite what was in the contract... turning energy supply into an economic weapon against us"
More supply chain risks were seen during Ukraine invasion:
"The military has to prepare or the governments for war, but I think the businesses have to prepare for war as well and it is the dependencies."
- IKEA found 25% of all raw materials came from Russia, and had to change overnight
- BMW had all their cables produced in two factories in Ukraine. In first 2 weeks of invasion, no more cables
"Going to the countries where labor is cheapest or certain expertise is the best or whatever the arguments are to go somewhere, then that is fine and understandable. But we have to become much more strategic. For toilet paper, I don't care where it's made. But for certain things, I think we have to consider bringing back production to Europe to North America to make sure that in our, let's say alliance territory"
This was a fantastic and sobering interview, a testament to the thinking going on at policy and corporate executive levels, on the economic realities of this age.
Other themes:
- solar is great, but introduces lots of potential supply chain risk to China — setting the stage for another energy disruption
- a call to action for corporations to prepare for massive disruptions to supply chains due to political and economic conflict, especially for things essential to society (e.g., energy, national security, etc.)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
I was talking with my friend @nealriley about how much fun I’m having with LLMs lately, including the project I've written about before to interpret screenshots of podcasts and YouTube videos. I made the observation that working with LLMs seems to boil down to a pattern of creating ETL pipelines, over and over again.
He quickly corrected me, suggesting, "Don't you mean ELT pipelines?"
I laughed at his curious and unexpected comment, which seemed unusually pedantic. However, I found myself thinking about his comment for days afterward.
It reminded me of a quip from Rich Hickey, inventor of the Clojure programming language: "Information is simple. The only thing you can do with information is ruin it."
In my experience, what Rich said is so true, and shows that Neal was absolutely correct. What I meant to say is "ELT," not "ETL." In other words, you should Extract, Load/Store, and then Transform data, in that order (ELT). Unless you love misery, you should not Transform before you Load/Store (ETL).
Over the last many decades, I've ruined data in so many many ways, for me, the “T” in “ETL” stands for tarnish, trash, taint, etc.
I’ve accidentally converted date-time strings incorrectly (turning them all into nils), I’ve discarded data fields that I thought were irrelevant, but years later, I wish I had kept them because it is priceless information, impossible to reconstruct. I’ve accidentally overwritten data, deleted data, and so many more terrible things, Thankfully, I've wiped those episodes from my memory, lest I start kicking myself again.
This is because I was Transforming the data (or Trashing it) before storing it.
One of the things I've learned from the Clojure and cloud community is the value of just storing data in its original form, often in storage buckets, deferring any transformation of the data to afterward, often at runtime (i.e., Extract, Load/Store, then Transform).
I almost made this mistake last week: I mentioned how I use @revai and @deepgram and to create podcast audio transcripts. At first, I converted the audio transcripts to match the format used by the python-transcript-api program. That way, I could use the same code to render the transcript, regardless of what the transcript source was.
This is when I once again learned that the only thing I could do to the data is to ruin it.
So instead, I saved the entirety of the transcript in its original format, including the full API return payload. That way, I’m preserving optionality, enabling myself to use that data in new ways in the future. By discarding the data, I destroy that optionality.
Today, I saw one such potential option: I realized that transcript diarization could be helpful (this is when transcripts detect when there are multiple speakers talking, and ideally labels them). The YouTube-transcription-api doesn’t support speaker labels, and I would have discarded that data to match it.
My lessons:
- Store your data in as close to its original form
- Defer data transformation to after you load/store it
Citations and resources:
Citation for Rich Hickey quote: "Information is simple. The only thing you can do with information is ruin it" - Rich Hickey (56:09)
A fun dialogue I had learning about ETL vs ELT with ChatGPT:
I'm finding using XTDB to be fantastic for these ELT pipelines: it's amazing to have an append-only database, where if you trash your data, you can just rewind the event log.
The benefits of ELT vs. ETL falls shows up in "Wiring the Winning Organization," co-authored with Dr. @StevenJSpear. Problems are much easier to solve when you can undo them — if you can't undo, you can't iterate and experiment, which also means you can't learn.
When experiments are highly consequential and thus expensive, you are very much in the Danger Zone.
Merely shifting Transformation to after Loading/Storing, we make problems dramatically easier to solve, enabling Winning Zone characteristics.
One of biggest learnings for me was the importance of architecture to get great outcomes, both in DevOps and in any engineered system.
I was trying to think of a great example of modularity, and started marveling at the USB interface.
What's amazing about USB?
The USB cable doesn't care what it's connected to — it connects computers to peripherals (e.g., printers, keyboards, mice, network adapters).
It's remarkable looking at the incredible explosion of innovation on both sides of that interface.
The types of things that USB connects has exploded, far beyond what could have ever been conceived of in 1996 (26 years ago).
On the computer side: it's not just PCs and Macs, but also cigarette lighters in cars, batteries, solar panels, but also fighter jets, ships, submarines.
...but I'm trying to refresh my memory on his teachings, and view it through the lens of how it informs how we create and exploit optionality, maybe against an adversary, but also on how one creates and operates systems.
(From Boyd to Steve Blank/Eric Ries & Dr. Carliss Baldwin)
But before I re-read the books on Boyd, I wanted to get a high-level survey of his material, and had so much fun watching this video — which included footage of multiplayer dogfight sims!
On Boyd: After Korean War, he became an instructor, he became known as "40 Second Boyd"
We speculate this aspect of Eroom's Law is what causes:
- the before vs. after state in Team of Teams
- waterfall -> DevOps
I think we even saw this happen in the COVID vaccinations (those that get 100% into people's arms vs 30%)
...and so many more domains!
Here are examples of where we think integrated problems solving was done superbly, resulting in amazing outcomes, vs those that were done poorly, resulting in disastrous outcomes.