I suggest that the problems are with the processes and systems. This is a thread on how VPs, CIOs, and CEOs might think differently, so that, eventually, the customer experience would be better. Surprisingly, this should also save money.
@ATT, I’m not asking for a point-by-point response to each of the problems. There will be a resolution on Tuesday, I’m confident. Part 2 of this thread talks about the underlying problems. I’m much more interested in what you and others might think about that.
My wife and I are moving from A to B, but will have both addresses for a period of time, so I called AT&T from address A to have internet and phone service set up for address B.
That was a few weeks ago, so I asked for the service to be turned on and the equipment (internet modem) to be sent before we arrived at B on Friday.
To get the new service for B, I had to open a new account, although I’ve been an AT&T customer for maybe 30 years. We have maybe three or four other accounts with AT&T, and they have nothing to do with each other. I’m not aware of a way to consolidate them to one account.
To get the new account, they had to check my credit, although I’ve been paying my bill automatically for maybe 15 years, on each of the three or four other accounts we have with AT&T.
So that anyone can access my credit, I have to unfreeze my credit with the credit reporting company (this process isn’t a great customer experience either, but that would be another thread).
To minimize that effort, I asked which credit bureau would be used, but the person handling my request couldn’t tell me, since that’s determined randomly, so I had to unfreeze all three of them and call back in.
When I called back in, I had to start from the beginning, rather than where we left off.
I often had to enter an account number to the automated system, but then I had to repeat it and maybe a pin number to whichever person I spoke to. This seemed to happen whenever I got transferred to another person, which happened often.
If we got disconnected, at least some of the time I had to start over, rather than being called back. If I was on hold, often the phone was silent, so I wasn’t sure whether something was still in progress.
Whenever I call in, I get a voice recognition system, which tries to narrow down my request to something that could be handled automatically. The system often doesn’t recognize what I’m saying, particularly if I’m using airpods instead of speaking directly into the phone.
There often aren’t numeric options for the menu items so that I can choose something by typing a number rather than making multiple attempts to be understood.
My general sense is that it takes one or two more attempts to get to a human than I think it should. I feel impatient.
About ten days ago I got an email confirming that my equipment was to be sent, which was good, but it didn’t say whether it was going to be sent to address A or address B. I called to verify that it was going to address B.
The tracking number for UPS wasn’t valid yet, perhaps because the equipment hadn’t yet been given to UPS, so UPS couldn’t tell me. I would need to get the information from AT&T’s systems.
I had to talk to 4 people to get an answer, as the various systems used by the customer service folks didn’t have the information. After hunting for a long time, the last person finally confirmed that the equipment was in fact going to address B. I was relieved.
Yesterday morning, my first morning at address B, I looked for the equipment. Not finding it, I tried the UPS tracking number again, which worked this time, and confirmed that the equipment was at address A.
Before I called service, I decided to check my phone line, and plugged my phone into all the jacks in the house. All were silent, but I don’t have a filter for the digital signal, so wasn’t sure what to expect.
I called in about the equipment delivery, and the person confirmed the error, and explained that she would transfer me to sales to correct the problem. The sales person then reconfirmed the problem.
I also asked about the phone line, and she said that silence is what would be expected, but that the line wasn’t live, because the original visit earlier in the week couldn’t be
completed without inside access while the owner was present.
completed without inside access while the owner was present.
Although I had originally been told the line could be turned on without inside access, there was apparently a problem in this case.
I had not been notified that there was a problem with the first visit, nor that I would need to schedule another visit. I was told that a note should have been left at the door, but I hadn’t seen one (although movers had been here, so the note might have been lost).
I asked why there was not also a notification sent to my email, since the whole point of the plan was to get access ready before we could arrive to read a note. I was told that the physical note is the only notification.
I was told that we could solve both problems at the same time, which was good news. When the technician returned to fix the line, they could also bring the equipment.
The only remaining problem was that the return visit could not be scheduled right now, because someone else was currently updating my record, and the record was locked against any other updates.
That someone was neither of the two people I had talked to so far on the current call and my previous calls were many days in the past. I asked to speak to a supervisor.
Before I was transferred to the supervisor, I asked if there was someone I could speak to about the whole experience I’ve had. I was given a phone number. When I tried it later, I was told it’s no longer in service.
The supervisor explained in more detail what she was attempting to do, which involved sending messages to the person who was in the record, and one or two people in that person’s management chain, none of whom responded (it was Saturday, so that’s not too surprising).
There is apparently no way to override the lock on my record, so the supervisor had to just keep trying. She called me back twice to give me an update, and volunteered to keep trying after her shift.
This morning, Sunday, on her next shift, she called me back to say the record was open, and she had scheduled the visit for the next open slot, on Tuesday, between 8AM to 6PM. The local office apparently can’t give more narrow slots.
I told her how much I appreciated her efforts to go the extra mile.
At the end of many calls to AT&T, the person will sign off by saying something like “Thank you for being the most important part of what we do.” That expresses a good sentiment, but it isn’t matched by what actually happens.
In part two of this thread, I’ll talk about what I believe are the underlying problems and opportunities here. This example is for AT&T, but I think the situation is similar at many large organizations.
For AT&T, none of this might be thought to be very important, since the real money is the $200+ that my family’s wireless bill runs per month, plus other smaller bills for our other accounts.
The thought could be that I might be annoyed, but that I won’t bother shifting everything to a new carrier. That has been true — I haven’t bothered — so far.
In suggesting how to get to a better customer experience, it might seem that we’re talking about spending more money than the way we are currently doing things. I suggest we could spend less money, not even counting the loss from frustrated customers.
I’ve worked in computer systems, mostly for big organizations, for almost 40 years. I recently finished a doctorate in mathematical behavioral science.
The problem that started me on the path to that PhD is why big organizations have such consistent problems writing big computer systems.
In the past twenty years there has been increasing interest in “agile” methods for writing software systems. By now, folks have been attempting to be agile long enough that we have come to see that results are uneven. Sometimes little seems to change.
I suggest that an important missing element is to better understand where we’re starting from, rather than just focusing on the new thing we’re moving to. Rather than go into the cognitive science, I’ll just dive into specifics.
A common metaphor is that a software system is a building. Under the Building frame, we expect to spend a lot of time designing the building (“architecture”) before we stick a shovel in the ground (“typing code”).
It takes a lot of bricks and concrete and steel to build a building, and one brick ("line of code") is a lot like another.
We also expect to spend a great deal of money building the building ("capital expense"), and a much smaller amount of money in each subsequent year maintaining the building ("operating expense").
Metaphors and frames are tools of our thinking. Like any tool, they aren’t right or wrong, they are just more or less useful. The Building frame is very useful when we are building a physical building. But I suggest that it is not very useful when we are writing software.
When we are writing software, we should consciously set the Building frame aside, and use other tools.
When building a real building, we are talking about physics and fixed structure — the foundation and framework are rigid. If we build a four story building, and later decide that the third floor is not serving our purposes, we can’t just take it out and put in a new third floor.
But with software, we can replace that third floor (layer), if we’re careful. The point of software is that it is changeable.
Also, a line of code is not a brick. We can’t just replace one line of code with another random line of code; the system will break.
Now, we tell ourselves, “I know that software isn’t *really* a building — that’s just a metaphor.” For the senior executives who are reading this — while you have been advancing in your careers, the cognitive science has also moved.
Over the past few decades we have come to understand that metaphors and frames are deeply held in our brains, and they affect our thinking in ways we are not always conscious of.
It takes real effort to bring frames and metaphors to the surface, and make them conscious tools of our thinking. In particular, it takes a lot of work to set them aside when they are not serving us well.
Another metaphor is that building a system is running a project. The Project frame is concerned with budgets, schedules, resources, and tasks.
The Project frame is essential in a large organization — it’s the best way we know of to make changes in how an organization works.
But the Project frame doesn’t serve us as well when the problem is writing and modifying software.
We end up having to make a business case to write or change the software. A business case requires a return on investment. Making the business case is itself a sizable effort, and tends to encourage that changes are bunched together as a package.
An easy way to think of such a bundle of changes is as a new system, as a new Building. The Building and Project frames reinforce one another, such that we get relatively infrequent big, expensive, tightly supervised projects.
In between, we have long periods where the only changes are those necessary to “keep the lights on” (Building frame thinking again).
The last metaphor is that writing software is production. This metaphor comes from economics, and introduces the More frame. More is more — more inputs gives us more outputs.
We each build the More frame as infants, even before we start talking — as water pours into the glass, we see the glass fill up. As sand pours onto the pile, we see the pile get bigger.
So without thinking much about it, we believe that if we spend more money and add more people, we’ll get more working software.
The More frame is useful across a wide variety of domains, but it is not useful everywhere. If we’re talking about a structured system, the More frame often doesn’t work. Consider a mosquito — a structured system.
Suppose we were making a 1950s B monster movie about an experiment gone bad, with mosquitos the size of dogs — wouldn’t that be scary? Well, only in the movies.
In the real world, a mosquito the size of a dog couldn’t fly. It probably couldn’t even breathe, because an insect’s respiratory system couldn’t get oxygen to all that volume. In this case, the More frame leads us to a failed system.
The More frame also works poorly when applied to the structured system of writing software. Each line of code is not just another brick in the wall of a building. Each line has to work with all the other lines, and each line does a different thing — that’s why it’s there.
Over the decades, what we’ve observed over and over again is that good big systems start from good small systems — “two guys in a garage”. The small system takes time — years — to build.
Only very gradually does the system get to a size where we can start to add more people productively. When we blow the mosquito up in weeks or months, it either fails outright, or is weak and fragile. By fragile, we mean that it’s hard to change safely.
Systems that are hard to change safely end up changing very slowly, if at all. We think “if it ain’t broke, don’t fix it.” That thinking goes along with the Building frame, where we’re stuck with the steel and concrete we laid down decades ago.
After a number of years, in frustration, we toss the entire system and rewrite it, as another Building. It has a pretty blueprint and graphics, and promises new features. But it’s still a Building.
Because we are desperate to move away from the problems of the old system, and because we’re a big organization, we throw money and people at it (More), supported by lots of people supervising (Project).
We get another weak and fragile system, if we get anything at all (sometimes we just fail). This is by far the most expensive, disruptive, and unsatisfactory way to write software.
To do better, I suggest we don’t need the next flashy technology, the next cloud or micro service. It might sound odd to hear that from someone who has spent decades in tech. But the underlying problems are not technical.
It’s that the BPM frames are very blunt tools when applied to writing software. They get results, the way a hammer can drive a screw into wood. It will work, sort of, but the results will be poor — expensive, slow, and fragile.
A more useful alternative metaphor is that a system is a product. Under the Product frame, we are constantly updating the software to respond to market demands and internal pressures.
The product is never finished, so making it simple to change safely is central to what we do. The Product frame matches better to the inherent attributes of software — with effort and discipline, software can be changed frequently, at low risk and low cost.
Another crucial frame we might just summarize as Software — a distillation of all we’ve learned about writing software in very small increments, released into production in days or hours, where all processes are automated — infrastructure, testing, deployment, backout.
This thread is addressed to executives of organizations who aren’t Google, Microsoft, Apple, Facebook, Amazon. Those organizations were started and are led by people who have absorbed the Software frame through their fingertips, after years at a keyboard.
For everyone else, the BPM frames are the default way of thinking about our business. Putting these frames aside when the problem is writing software is a simple idea, but it is not easy. The BPM frames permeate our thinking.
To understand the challenge, consider the Christmas tree. Decorated evergreen trees only became associated with Christmas in the last few hundred years, in northern Europe, far from Jerusalem. These days, it is difficult to think of Christmas without a tree coming to mind.
A Christmas tree is a conceptual blend — it brings two or more ideas together through some common association, regardless of their logical relationship. We do this all the time, often without being conscious of it.
A popular blend when thinking about software these days is the idea of “scale”. Scale is a blend of the Software and More frames — it suggests that a useful and important goal of a big organization is doing everything that a small organization can do, but just more so.
Scale (the More frame) is important in some areas of IT, for example, when we are trying to ramp up to handle more volume, more customers. But writing software doesn’t scale, any more than mosquitos scale.
It is very hard to let go of the BPM frames, of the idea that a big organization should be able to do anything a smaller organization can do, only more so.
The BPM frames aren’t just ideas. They are embedded in our compensation systems, our accounting systems, and the minds of our stakeholders and boards of directors. Letting go of the BPM frames for writing software is a simple thought, but is very difficult to accomplish.
I suggest that the BPM frames are part of what leads to the kind of customer experience in the first part of this thread. I have no doubt that within AT&T there are many people who have wanted to fix each of the problems I described.
Few of these are likely to be major changes, but even small changes can be overwhelming to accomplish when our only tools for thinking about them are the BPM frames, and the fragile systems produced under the BPM frames.
It’s possible to do better, and even save money. But doing better starts in the minds of our executives.
There’s a lot to expand on here, but this is enough for now.
For easier reading, here's the threadreader version on customer experience and the Building, Project, and More frames: threadreaderapp.com/thread/1277334…
