The governor of New Jersey called explicitly for COBOL programmers to help with the development of the unemployment system:
This is how the software environmentalism crisis looks like. An explanatory thread. 1/
This is how the software environmentalism crisis looks like. An explanatory thread. 1/
In the software development industry, we focus exclusively on building systems. The conversations are dominated by new languages, new frameworks, new deployment options. And the body of software grows exponentially. At the same time, we are unable to remove old systems. 2/
From this perspective, our software industry behave not unlike the plastic industry: we focus on building without any regards to recyclability. That is not sustainable. And, given the breadth of software’s reach, it is not responsible either. 3/
The call for COBOL programmers by the governor of New Jersey takes people by surprise. And so does the sudden growth of #COBOL popularity:
4/
4/
But, this is merely a symptom. The underlying problem is systematic. COBOL is not a problem per se. The problem is that we associate COBOL with extinction, while it is still the most used programming language in the world.
5/
5/
We might think that changing a system is just a matter of will. Interestingly, in the case of the New Jersey unemployment system, there were many signs. For example, take a look at this one:
6/
6/
I would be surprised if the people that manage this core system did not know that depending on a technology considered outdated poses a great risk. I’d also be surprised if they didn’t try to address it, and fail for one reason or another.
7/
7/
I am seeing this with every company we interact. Everyone seems to have least one large skeleton in a closet right at the core of the business that they simply cannot get rid of.
8/
8/
When I ask people at various conferences to raise their hand if their company relies on at least one mainframe system, the large majority does. This is not a local problem. It’s a systematic one that requires a systematic rethinking of our work.
9/
9/
We need to change the narrative. Recyclability must be embedded into how we build our systems.
10/
10/
The term “software” was born to contrast “hardware”. At that time, hardware was as large as a building and software was soft and easy to replace. Now, we are in a different world: hardware fits in my pocket, while we are drowning in software we cannot move.
11/
11/
When developing systems, people often focus on current or immediate functionality. That is short sighted and dangerous. Functionality is indeed a business asset, but the ability to adapt to changes in the surrounding environment depends on the non-functional parts.
12/
12/
To recycle a system implies taking it apart and refurbishing it for other purposes. However, before we can take it apart, we first have to understand the parts.
13/
13/
Interestingly, the activity of understanding a system happens to be the single largest expense we have, with developers alone spending 50% or more of their time reading code. Yet, nobody talks about how this time is spent. That’s the first thing we should change.
14/
14/
We need to start talking about how we read code. Better yet, how we should not read code.
15/
15/
We read code because we want to understand enough to know what to do next. This is called decision making. Given that we spend most of our energy doing that, it would be more interesting to define our work as being primarily a decision making one rather than construction one.
16/
16/
As the goal is decision making, reading is just the strategy through which we extract information out of our system. It also happens to be the most manual and error prone strategy.
17/
17/
When I ask technical people whether they have pictures about their systems, everyone says yes. But, when I ask whether they believe those pictures are 100% accurate, most people say no. That’s because the pictures were created manually.
18/
18/
A manual picture or report shows either a wish or a belief. Both are essential for humans, but neither should be used as basis for decision making.
19/
19/
Decisions are hard enough as it is. Often uncertainties in the environment requires probabilistic thinking. However, the reality of our own system is not in this category. This must become a certainty.
20/
20/
The sheer size of our systems far exceed the ability of humans to grasp without summarization. A picture does tell a thousand words, but those words must be accurate.
21/
21/
Accurate pictures are still not enough. For a picture to be effective, it must be specific to the question we want to answer.
22/
22/
Software systems are likely the most complicated structures we have ever created as humans. To this end, it is impossible to predict specific questions. We can predict classes of questions but not specific ones.
23/
23/
That is why pictures have to be created after the question was made explicit. The only way this can happen is when this ability is inside the team. Given its impact, this ability is not merely a technical one. It has high level business strategic importance.
24/.
24/.
A good picture or narrative can make humans grasp a problem orders of magnitude faster. A decade ago this was less obvious, but by now we have ample evidence in all the efforts around data science and the impact it has on business decisions.
25/
25/
As everything about our systems is data, the same principle applies to software development too. We just have to think of our job through this new lens. And this might be less difficult than expected.
26/
26/
Just think of this: most systems take data, model it, manipulate it and present it to users that make decisions fast without ever seeing the raw data. So, we, software people, possess this magic skill of transforming data into information to speed up decision making.
27/
27/
The skill is already present inside our teams. We just have to apply it to our own work!
28/
28/
Every time we read, we imply that the only way to answer the question is manually going through raw data. However, through our work and that of others, we showed that most software assessment is rather decomposable as a search problem.
29/
29/
That might sound academic, but it’s really not. Consider this: when a developer addresss a data problem in the database, she first writes a query. When the same developer addresses a problem in code, she scrolls. Both are data problems. Only the mental models are different.
30/
30/
We should observe that there is a deep correlation between the interface of our tools, and the mental models they induce. A database tool has a big query box at the top. People query. A typical development environment has a big editor in the center. People read.
32/
32/
McLuhan & Culkin told us that we shape our tools, and thereafter our tools shape us. We should carefully choose the characteristics of our tools because they decide how we think. Our tools, and not only our languages, are essential. The stakes are too high to disregard this. 33/
How would this look like in practice? Consider this case study of reasoning about evaluating @Deprecated classes in a Java system:
34/
34/
In our experiments, we ask devs to estimate the time needed to go over a given system to find which deprecated classes they can remove and what to do with the ones that cannot be removed right now. For that specific system, the average estimation is 4h, the minimum being 2h.
35/
35/
Then we show how the same problem can be addressed better and more accurately in a matter of minutes. This is an order of magnitude difference when dealing with a simple scenario. The difference only gets larger for more broad problems and larger systems.
36/
36/
But, the more interesting part is not the tool. The specific tool used is relevant to show what is possible. The most interesting part is that people can understand what the tool does, even though not necessarily the language details, in minutes!
37/
37/
There is no conceptual learning curve because developers have already done the same thing many times, only on other data. This ability is essential and it is reusable.
38/
38/
Tools are essential in software development. The key characteristic is that they should be customizable inexpensively while working on the problem. We saw they should be moldable.
39/
39/
Every time you replace manual work with automation, the real value is extracted when you change the business processes. The same applies here. We have seen historically that when tests became code, we shifted towards various forms of test-driven development.
40/
40/
Once infrastructure became code, we reshaped companies around #DevOps.
Now, tools must also become code. And this leads to #MoldableDevelopment.
41/
Now, tools must also become code. And this leads to #MoldableDevelopment.
41/
This thread is already way too long. Does this cover the whole space? No. Do we have all the answers? Certainly not. But, we do have the beginning of the conversation. And this is a conversation we must have.
42/
42/
Our kids will only know a world based on software. We already messed up the real world for them, and that is certainly something we must address.
43/
43/
While addressing the real world, we should not create a larger problem. The software world is in a crisis of environmental proportions, too. Like with the real environment, it is addressable. We just must see it for what it is, and make it an explicit subject of conversation.
44/
44/
