Thread by @MelanieCebula

I'll be live tweeting #KubeCon #CloudNativeCon, and taking this year off for speaking. Excited to sit back and actually watch the content 😊. What talks are you attending?

@Lemonjet here to give a keynote on K8s @ Apple. They have MASSIVE data center scale. Looked to K8s for the pluggability, extensibility, and ecosystem. Unsurprisingly, they had to consider the learning curve and platform support to drive adoption. #kubecon #CloudNativeCon

Apple started by breaking down different users and workloads. Application developers, SRE (Note: easy to forget that infra teams are also your customers!), hardware, machine learning / batch, and finance / payments jobs. #kubecon #CloudNativeCon @Lemonjet

Tech challenges include providing both cluster-as-a-service, namespace-as-a-service, providing open source tooling, as well as CRDs (these are HUGE for custom config/apply logic, needed to integrate with legacy/internal systems) @Lemonjet #CloudNativeCon #kubeconNA

Speaking of tech challenges, webcasting to this many people appears to be stressing the webcast infra! Wonder what tech it's using? 🤔 #kubecon #CloudNativeCon

Focus on security and isolation in multi-tenancy: microVMs and virtual clusters. I think using microVMs will become increasingly popular for these reasons! @Lemonjet #CloudNativeCon #kubeconNA

Not surprising (to me): K8s does *not* get you out of capacity planning and cost efficiency! Apple created a standalone team for these efforts. @Lemonjet #CloudNativeCon #kubeconNA

I love the focus on both open source and inner source! I think integration is *the* massive challenge for a footprint like Apple's. Great job @Lemonjet #CloudNativeCon #kubeconNA

Now, CNCF project updates with @ccaramanolis, focus on end user stories (Note: proof that people are actually using this stuff 😉). Guessing a potential theme: ARM support? #kubeconNA #CloudNativeCon

Starting with falco for cloud-native runtime security. Uses eBPF and all the latest greatness. Used in production by @ShopifyEng Anyone else using it? @ccaramanolis #CloudNativeCon #kubeconNA

thanos (highly available prometheus setup), incubating, lots of contributions, integrating with cortex project @ccaramanolis #CloudNativeCon #kubeconNA

rook (cloud-native storage) is used by pacific research platform, which includes my alma mater @UCBerkeley. Storing up to several PBs of data in different regions. Always impressed by research institutions willingness to try out the cutting edge of tech! @ccaramanolis

@vitessio (scale SQL) running on k8s. Has anyone been able to adopt this tech on K8s without at least some drama? @SlackEng able to migrate 70%, but remaining 30% is the crux. Need to get very clever and creative to migrate remaining workloads (now 99% traffic). @ccaramanolis

VolcanoEngine contributing to KubeFlow-- always interesting to see how K8s and ML technologies are used together at scale. I don't think we're quite at a "unified" Cloud Compute Layer, but K8s sure is trying to get us there! #CloudNativeCon #kubeconNA

@stephenaugustus giving project updates on @kubernetesio starting with @CloudNativeFdn's response to COVID-19, racism, trolls, and more! #kubecon

@stephenaugustus giving SIG Updates-- always a great way to know where the hard problems are in K8s land 😊 Usability, multicluster, storage... #CloudNativeCon #kubecon

@ccaramanolis has the cutest dog in the webcast, my closing remarks are "can we have more dogs in the keynotes pls" 🐾 #CloudNativeCon #kubecon

Continued adoption of @containerd by Apple, IBM, and Alibaba folks @estesp
Increased support in: cloud providers, k8s as a service, docker, dev tools (ex: buildkit)
I'm personally excited by the larger-footprint end-user adoption of this technology #CloudNativeCon #kubecon

@rothgar taking us through a real-life example of running movie-streaming infra on k8s (alongside monitoring, scheduling, hardware/resource usage, cluster scaling, cloud storage, and other concerns). Enjoying the home movie vibe of this one! #CloudNativeCon #kubecon

appreciating the low-level topology updates mentioned in containerd talk(CPU core scheduling, NUMA, L3 cache, etc). I also experimented with CPUManager and TopologyManager; now there's NRI (NodeResourceInterface) for hooking into container lifecycle hooks #CloudNativeCon

@rothgar talking about 20/80 (20% content viewed 80% of the time) and using a CDN to cache content, deciding what to store locally and what to synchronize; exactly the sort of bag of tricks infra teams use at scale! #CloudNativeCon #kubecon

@containerd CRI/containerd code merge, and NRI (Node Resource Interface) support in 1.5, needs more documentation and encouraging security adviser/support. I appreciate how containerd focuses on all the important stuff 👏 #CloudNativeCon #KubeCon

Alyssa Wilk discusses how after working on Google's proxy for 10+ years, she switched to helping get @EnvoyProxy up to google standards. That's quite the endorsement! #CloudNativeCon #kubecon

@mattklein123 built twitter's edge proxy, which is what lead to @EnvoyProxy, which today is used in a lot of different domains (service mesh to API gateway) #CloudNativeCon #KubeCon

Alyssa jumping into migration pain with @EnvoyProxy, which assumes you're using HTTP 2, but not the reality with legacy workloads and issues with standardization / following the spec #CloudNativeCon #KubeCon

@mattklein123: theoretically everything lives on standards, but everyone runs into these (migration) issues. @EnvoyProxy alleviates this with good metrics, monitoring, tracing, etc. Focusing on the observability and diagnostics when things break #CloudNativeCon #KubeCon

@mattklein123: his advice "don't do any of this unless you actually have to"

*gestures at entire CNCF ecosystem*

#CloudNativeCon #KubeCon

@linsun_unc: even @EnvoyProxy project moved back from microservices to monolith, and are a lot happier now (yes, thanks for saying this!!!)

#CloudNativeCon #KubeCon

@linsun_unc: more sidecars -> more connection pools -> more chance of things going wrong (ex: 503s very common)

@mattklein123: chain of proxies, and often timeout config gets messy and confusing across all the layers (ex: HTTP 1.1 doesn't handle this well)
#CloudNativeCon

Request: Can someone draw all of the proxy layers as a multi-headed beast? #KubeCon #CloudNativeCon @mattklein123 @linsun_unc @CloudNativeFdn

@mattklein123: "With great observability comes great power and responsibility"

People tend to hyperfocus on small, low-level issues, and not the big picture of the reliability of the entire system

Operational experience still necessary for diagnosing minor vs major issues

@mattklein123 looking at entire menu of options: vendor vs open source vs internal, be realistic about actual costs

Note: This is why I don't like blanket statements like "Not Invented Here" or "Build vs Buy". Evaluate the tradeoffs for every decision, and revisit them later

@gabbifish and @pwittrock gives us an overview of CLIs in the K8s community: starting with k8s configuration abstraction
templating (helm, ytt)
YAML composition (kustomize)
DSL (cue, isopod)
general purpose languages (pulumi)
#CloudNativeCon #kubecon

Note: you can chain CLIs (ex: Helm + Kustomize) to get both abstraction & variance! @gabbifish @pwittrock #CloudNativeCon #kubecon

DSL options like Cuelang provide a one-stop shop for abstraction, variance, and cross-cutting concerns: provides defaults and types using a patch-type file, and can throw validation errors for type mismatch @gabbifish @pwittrock #CloudNativeCon #kubecon

Options that use general purpose languages like to write config as code (ex: pulumi). Imperative logic to determine values for different fields, able to use general-purpose libraries and linters @gabbifish @pwittrock #CloudNativeCon #kubecon

And finally, k8s controller-esque option (Ex: Kpt): read low-level resource and populates/promotes it to a high-level resource (ex: using a starlark script). Leverages K8s composable declarative format, but new and experimental. @gabbifish @pwittrock #CloudNativeCon #kubecon

Want to keep it simple? Use Templating + YAML Composition

Want more flexibility? DSL (declarative) & General-Purpose Languages (imperative)?

Mix and match logic on various types of inputs? Controller-esque

@gabbifish @pwittrock #CloudNativeCon #kubecon