Thread by @stevesi

"What is the future of operating systems?" Is a great question that comes up quite a bit (like @Stripe yesterday). "Future of" is always tricky but one thing is always the case in software, which is everything interesting in tech moves "up the stack". Thoughts… 1/

2/ In earliest days of microcomputers (PCs before PCs) it was not uncommon for those interested to spend most of their time debating the merits of various processors. This is from a 1977 debate.

3/ At the time many believed that computing would evolve to have many competing "computers" built on different platforms. But this is never a sustainable state. Why? Because to move forward requires engineering abstractions, and abstractions mean interesting stuff moves up.

4/ By 1981 consumer PCs became interesting (thanks to Apple, Tandy) and then IBM introduced the PC. Many think it was IBMs "footprint" that made the PC what it is. Maybe, but it was also the desire for creative people to stop worrying about details of processors and OS variety?

5/ The abstraction of the OS from competing chipsets created a vibrant world of software choices. While there were two main OS competitors (Apple and Microsoft) they were similar enough for a while that software ran on both. Mac and Win Excel 3.0 c. 1990.

6/ Some thought this would be a steady state. But a growth market abhors a steady state. People are inventing and re-inventing, driving the underlying layers to provide new services and to rethink abstractions. And competition at OS layer drives innovation.

7/ That competition resulted in a wave of graphical interfaces. From 1985-95 the industry was awash in various "GUI" approaches. Again, many thought the future looked like making a bet from 1 of many graphical systems. Windows, Mac, OS/2, NewWave, NeXT, Amiga, VisiOn, GEM…

8/ Once again the industry and market created a center of gravity that allows everyone (hardware, peripherals, apps, systems) to innovate while holding a few things constant. This is actually good for everyone. Honest.

9/ Debates about which abstraction is "best" are often shown to be irrelevant or details that ultimately are deciding factors are often not what technologists would pick. Mac was superior in design, but breadth of PC hardware and ecosystem prevailed and became "best."

10/ Even though the Mac was "better" in some ways, the market pulled the PC through to a stronger product-market fit. That, for lack of a better word, made the PC "best". It would never settle the debate but if the score is number of units then the PC became best.

11/ This cycle is not constrained to whole systems. Even within a technology system this cycle of diversity to consolidation around a "winner" happens constantly often due to externalities. Networking went through this as well.

12/ While in PCs the "battle" between Netware and Windows was legendary ("better dead than red" the LanMan team would say) the internet and TCP/IP came along. For a while TCP/IP was yet another choice. Too much complexity that slowed innovation.

13/ Here’s a long list of TCP/IP stacks for Windows 3 c '95. IDC estimated TCP/IP to grow to $1.7B in 1997. Windows NT was *designed* to deal w/network diversity. But really was it ever sustainable to have something so deep in an OS have diversity and complexity that results?

14/ Early in innovation, products tend to be similar and by and large those building on the layer can abstract it (eg network drivers, x-platform apps). That is what happened with Android and iOS. Products start with the same needs (WWAN, basic touch, etc.) but rapidly diverge.

15/ Innovation is two sided. The providers of abstraction are innovating as are those building on top. As much as everyone wants "stability" as a platform provider, stability is also stagnation. This causes the abstractions to diverge and working across them much more difficult.

16/ In the late 90s Windows and Mac diverged to a point that it became nearly impossible to keep the code bases in sync while also functioning like well-behaved apps. We see this today on mobile. Now mobile is such a huge market that stability might look like a duopoly.

17/ That happens too. Many abstractions can support two, but not forever. That’s because at some point there are tectonic shifts in abstractions. That’s what the cloud is. Example: OracleDB and SQL were a duopoly (note usually it is like 80/20). But cloud upends everything.

18/ The cloud is currently in state where many believe we in some natural state of multiple players. As a reminder, many observers (analysts, stock) love this situation. Like IDC and TCP/IP.

The question for "future of OS" is what is the abstraction we are talking about?

19/ Today’s clouds, however, look to me a lot like debates over chipsets in 1977. Analysts call this "IaaS" but that confuses everyone as much as fighting over instruction sets.

The real question is what are the key abstraction layers where economic leverage arises.

20/ Some believe the next platform will be defined by some sort of layer above, like an OS but cloudy. This is entirely natural. In fact the main "IaaS" providers will almost certainly innovate in these areas and third parties will do these as x-platform.

21/ This will happen and it will become increasingly difficult to be "multi-cloud" no matter how much anyone wants to be. But this will likely be a database-like "market" and some logical subset will emerge, much to the chagrin of clouds and "apps" will be what matter more.

22/ I believe the real "next OS" then will be much more abstract at the services layer. Just as for many the next OS of the PC ended up being the browser and Office, the next OS for the cloud will be APIs that do really substantial work.

23/ Right now that seems like a fragmented and pretty narrow spaces (solutions) with a lot of players. It kind of is. But the companies that build services that abstract massive complexity will create whole ecosystems around that complexity.

24/ Remember there were dozens of word processors, browsers, and TCP/IP stacks. There’s nothing about this leading to a conclusion that the BIG providers of infrastructure end up dominating in this space. eg who dominated DB apps?

I would argue they will be challenged to do so.

25/ To them spaces seem too fragmented to them AND they do not want to alienate either customers or partners. They’d rather have many APIs for same thing than risk picking or building a "winner" too soon. This is the real challenge of building a platform during tectonic shifts.

26/ Customers (devs, ecosystems) begin to demand a coherent platform, not just choice. Choice is expensive relative to innovation. Most don't want to innovate at a low level and want to offer domain specific or advance capabilities. Eg most don't want to build a browser.

27/ Today to platform vendors APIs look like "middleware" and focusing on supporting lots of middleware or tools for managing middleware or tools for managing at a lower level seem much more comfortable.

28/ To me it means there is a lot of room for the next platform to emerge outside of what today is thought of as the cloud platform provider.

29/ If you think that is crazy, ask yourself where the PC platform came from compared to the dominate player, or the leading database (not IBM either), or where the browser came from, or where the leading mobile platforms came from. // END

PS/ This thread is also why "market definition" in software, which essentially exists to continuously re-abstract and simplify so we can do more is so difficult. Without a cycle of invention-fragmentation-reabstraction we would still be arguing over chips and network protocols.//