Kubernetes Borg/Omega history topic 10: In honor of #KubeConEU and the 5th anniversary of open-sourcing Kubernetes, I’ll add more perspective from the Borg and Omega teams to the origin story
Internally, Google puts a lot of emphasis on both resource efficiency and engineering efficiency. For both reasons, back in June 2013, a few months before GCE was ready to GA (cloudplatform.googleblog.com/2013/12/google…), the Borg and GCE teams started to work more closely to improve both
The initial focus gravitated towards directly supporting features needed by Cloud in Borg so that Cloud wouldn't need to work around the the lack of those features (see my previous comment on 2-level scheduling: )
Google also spends a lot of energy constantly mitigating entropy in its internal software and infrastructure. The monorepo is one mechanism for this. It also launches many efforts to “unify” or “converge” multiple systems that co-evolve to do similar things.
In that context, 2 months later, the Unified Compute Working Group was formed by Google Cloud and Google's internal infrastructure group, "TI", which included Borg. The goal was to develop a proposal for a "compute platform" that could be used by both Cloud and internal customers
It was obvious that VMs would be too cumbersome and inefficient and PaaSes of the time weren't versatile enough to run a wide range of internal services, such as web search and Gmail. We needed a platform that was more like Borg, that was based on containers.
There were discussions regarding how compatible it should be with App Engine and with Borg. Docker, buildpacks, and Omlet (a new node agent under development to replace Borglet) were compared. Early discussions presumed a managed service, like GCE, App Engine, and Borg
In September 2013, viewpoints of 9+ WG participants were collected and composed into a "Unified Compute PRD", focused on serving workloads (e.g., rather than batch). That was the first time I was aware of the term "Container as a Service" being used
In October, subgroups of the WG were formed to focus on key problems, including a container management API subgroup. In November, we pulled in more people from Borg and from Cloud to hash out a number of API details. In December, an API proposal was presented to the full WG
At the same meeting, a proposal was made for what became the App Engine Flexible Environment (cloud.google.com/appengine/docs…), and a proposal to build an open-source container platform, so that we wouldn't be "Hadooped" by other OSS projects
That OSS container platform was Project 7. After, there were several proposals from both the Borg side and Cloud side to build products with compatible APIs. Collaboration with the Borg team deepened. Borglet team members started to work on libcontainer for Docker in April 2014
It soon became clear that other Borg team members (me @thockin @erictune4 Dawn Chen @originalavalamp @davidopp @vishnukanan) should work on the open-source project to design and develop the Borg-like functionality. We deeply believed in the potential value to external users
So Google Cloud created Kubernetes in part because Google needed it, and we believed others needed it also. Looking at the other solutions available at the time (e.g., github.com/tsuru/docker-c…, github.com/signalfx/maest…) and since, we made the right call
Ah, I found that @jbeda covered some of this background from the Cloud side in his podcast: kubernetespodcast.com/episode/012-ku…
Dawn spoke about her involvement in her podcast: kubernetespodcast.com/episode/022-si…
And other previous origin stories and retrospectives: cloudplatform.googleblog.com/2016/07/from-G…
kubernetes.io/blog/2016/07/h… kubernetes.io/blog/2016/07/t… kubernetes.io/blog/2016/07/o… kubernetes.io/blog/2018/06/0…
kubernetes.io/blog/2016/07/h… kubernetes.io/blog/2016/07/t… kubernetes.io/blog/2016/07/o… kubernetes.io/blog/2018/06/0…
There were some posted after the OSCON Most impact award also: kubernetes.io/blog/2018/07/1… cloudplatform.googleblog.com/2018/07/kubern… kubernetes.io/blog/2018/07/2…
