Gene Kim Profile picture
Sep 1 5 tweets 2 min read Read on X
Amazing and sobering WSJ interview of Dutch Admiral Bauer, NATO's top military officer, on supply chain risks.

"We've long thought that mutual economic relationships between nations would prevent war. We thought we were buying our gas from a company. Turns out that wasn't true.

"Putin told Gazprom to turn the [gas] off, despite what was in the contract... turning energy supply into an economic weapon against us"

More supply chain risks were seen during Ukraine invasion:

"The military has to prepare or the governments for war, but I think the businesses have to prepare for war as well and it is the dependencies."

- IKEA found 25% of all raw materials came from Russia, and had to change overnight
- BMW had all their cables produced in two factories in Ukraine. In first 2 weeks of invasion, no more cables

"Going to the countries where labor is cheapest or certain expertise is the best or whatever the arguments are to go somewhere, then that is fine and understandable. But we have to become much more strategic. For toilet paper, I don't care where it's made. But for certain things, I think we have to consider bringing back production to Europe to North America to make sure that in our, let's say alliance territory"

This was a fantastic and sobering interview, a testament to the thinking going on at policy and corporate executive levels, on the economic realities of this age.

Other themes:

- solar is great, but introduces lots of potential supply chain risk to China — setting the stage for another energy disruption
- a call to action for corporations to prepare for massive disruptions to supply chains due to political and economic conflict, especially for things essential to society (e.g., energy, national security, etc.)

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Gene Kim

Gene Kim Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @RealGeneKim

Jul 19
I'm in awe of the scale of the Crowdstrike / Windows BSOD issue.

Here are the most startling images I've seen morning.

Let's start with this: at 10pm PT yesterday, famous @troyhunt notices that something odd is happening to Windows systems:

12-hour timelapse of American Airlines, Delta, and United flights:

Not sure if this is true, but @ErrataRob says that the skies over US have not been this clear of airplanes since the 9/11 ground stop.

Read 65 tweets
Mar 18
I was talking with my friend @nealriley about how much fun I’m having with LLMs lately, including the project I've written about before to interpret screenshots of podcasts and YouTube videos. I made the observation that working with LLMs seems to boil down to a pattern of creating ETL pipelines, over and over again.

He quickly corrected me, suggesting, "Don't you mean ELT pipelines?"

I laughed at his curious and unexpected comment, which seemed unusually pedantic. However, I found myself thinking about his comment for days afterward.

It reminded me of a quip from Rich Hickey, inventor of the Clojure programming language: "Information is simple. The only thing you can do with information is ruin it."

In my experience, what Rich said is so true, and shows that Neal was absolutely correct. What I meant to say is "ELT," not "ETL." In other words, you should Extract, Load/Store, and then Transform data, in that order (ELT). Unless you love misery, you should not Transform before you Load/Store (ETL).

Over the last many decades, I've ruined data in so many many ways, for me, the “T” in “ETL” stands for tarnish, trash, taint, etc.

I’ve accidentally converted date-time strings incorrectly (turning them all into nils), I’ve discarded data fields that I thought were irrelevant, but years later, I wish I had kept them because it is priceless information, impossible to reconstruct. I’ve accidentally overwritten data, deleted data, and so many more terrible things, Thankfully, I've wiped those episodes from my memory, lest I start kicking myself again.

This is because I was Transforming the data (or Trashing it) before storing it.

One of the things I've learned from the Clojure and cloud community is the value of just storing data in its original form, often in storage buckets, deferring any transformation of the data to afterward, often at runtime (i.e., Extract, Load/Store, then Transform).

I almost made this mistake last week: I mentioned how I use @revai and @deepgram and to create podcast audio transcripts. At first, I converted the audio transcripts to match the format used by the python-transcript-api program. That way, I could use the same code to render the transcript, regardless of what the transcript source was.

This is when I once again learned that the only thing I could do to the data is to ruin it.

So instead, I saved the entirety of the transcript in its original format, including the full API return payload. That way, I’m preserving optionality, enabling myself to use that data in new ways in the future. By discarding the data, I destroy that optionality.

Today, I saw one such potential option: I realized that transcript diarization could be helpful (this is when transcripts detect when there are multiple speakers talking, and ideally labels them). The YouTube-transcription-api doesn’t support speaker labels, and I would have discarded that data to match it.

My lessons:

- Store your data in as close to its original form
- Defer data transformation to after you load/store it
Citations and resources:

Citation for Rich Hickey quote: "Information is simple. The only thing you can do with information is ruin it" - Rich Hickey  (56:09)

A fun dialogue I had learning about ETL vs ELT with ChatGPT:



I'm finding using XTDB to be fantastic for these ELT pipelines: it's amazing to have an append-only database, where if you trash your data, you can just rewind the event log.



The benefits of ELT vs. ETL falls shows up in "Wiring the Winning Organization," co-authored with Dr. @StevenJSpear. Problems are much easier to solve when you can undo them — if you can't undo, you can't iterate and experiment, which also means you can't learn.

When experiments are highly consequential and thus expensive, you are very much in the Danger Zone.

Merely shifting Transformation to after Loading/Storing, we make problems dramatically easier to solve, enabling Winning Zone characteristics.

cc'ing @fistsOfReason, because I'm proud of using the word "optionality," @refset because I love @xtdb_com, @enigma2a .infoq.com/presentations/…
chat.openai.com/share/aca1afd1…
v1-docs.xtdb.com/guides/quickst…
Cc @mtnygard because he’s so smart, will surely have some insights, and has long achieved data transcendence.
Read 5 tweets
Jun 23, 2022
One of biggest learnings for me was the importance of architecture to get great outcomes, both in DevOps and in any engineered system.

I was trying to think of a great example of modularity, and started marveling at the USB interface.
What's amazing about USB?

The USB cable doesn't care what it's connected to — it connects computers to peripherals (e.g., printers, keyboards, mice, network adapters).

It's remarkable looking at the incredible explosion of innovation on both sides of that interface.
The types of things that USB connects has exploded, far beyond what could have ever been conceived of in 1996 (26 years ago).

On the computer side: it's not just PCs and Macs, but also cigarette lighters in cars, batteries, solar panels, but also fighter jets, ships, submarines.
Read 33 tweets
Jun 19, 2021
A genuine question:

Can someone come up with an example NOT RELATED to software that has this property:

As time progresses, it becomes more and more difficult to add things, even when only one person is responsible for entire system.

(And then explain why this happens?)
In a call with @mik_kersten and @stevenjspear, I was amazed that I couldn't verbalize why this happens.

Ultimately, I'm hoping that this will inform why this phenomena observed by @johncutlefish happens:

@mik_kersten @StevenJSpear @johncutlefish In that tweet, @johncutlefish describes a condition where the same class of feature takes ten times longer to implement, at time t vs t + 3 years.

I'm certain that this phenomenon must exist in other domains:

- architecture: it's easy to add a room to a house, until it isn't
Read 6 tweets
Jun 12, 2021
Taking notes from this great short video on USAF Colonel John Boyd, his Energy Maneuverability theory, OODA loops, and more.



I've read the awesome book "Boyd: The Fighter Pilot Who Changed the Art of War" decades ago, but...

amazon.com/Boyd-Fighter-P…
...but I'm trying to refresh my memory on his teachings, and view it through the lens of how it informs how we create and exploit optionality, maybe against an adversary, but also on how one creates and operates systems.

(From Boyd to Steve Blank/Eric Ries & Dr. Carliss Baldwin)
But before I re-read the books on Boyd, I wanted to get a high-level survey of his material, and had so much fun watching this video — which included footage of multiplayer dogfight sims!

On Boyd: After Korean War, he became an instructor, he became known as "40 Second Boyd"
Read 33 tweets
Jun 11, 2021
Eroom's Law:

Since 1950s, for every $1B spent on pharmaceutical R&D, the # of drugs that make it market halving every 9 yrs! (The opposite of Moore's Law)

@StevenJSpear & I speculate that major cause is the # of disciplines of which integrated problem solving must occur!
We speculate this aspect of Eroom's Law is what causes:

- the before vs. after state in Team of Teams
- waterfall -> DevOps

I think we even saw this happen in the COVID vaccinations (those that get 100% into people's arms vs 30%)

...and so many more domains!
Here are examples of where we think integrated problems solving was done superbly, resulting in amazing outcomes, vs those that were done poorly, resulting in disastrous outcomes.

cc @MatthewEGunter @StevenJSpear
Read 10 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(