Thread by @denis_makogon

The reason why I love and hate #Kubernetes is it actually good platform to host scalable apps. But the road from developing the app to making it scalable is so painful. Here are few-reasons-why-thread:

#Kubernetes in its bare configuration can only host containers and let you talk to them. Include config maps, volumes, RBAC, etc.

If you ever wanted to scale your app on #Kubernetes you probably heard of MetricsServer - it can collect standard metrics like CPU and RAM of every pod/container.

It’s not a part of #k8s distribution - you need to install it separately.

#HorizontalPodAutoscaler requires MetricsServer to do the actual scaling. The most strange thing that a Deployment resource cannot scale in/out based on ration of dead or alive to all pods. This type of a criteria considers as a custom metric - you need a new API resource for it!

How often people do scaling on #Kubernetes? They use #Prometheus and the adapter! So, your app exposes metrics endpoint, prometheus scraps them and stores them in TSDB. The adapter is used to expose those metrics as custom metrics for scaling that HPA can read and use to scale.

So, until #Kubernetes v1.21 typical cluster setup looks like:
- #Calico for network policies.
- ingress controller for reverse proxy
- metrics server for standard metrics
- #prometheus for metrics collection
- adapter for custom metrics API resources

…and so on.

We found ourselves managing lots of additional software at scale without having event started to deploy the actual app on #Kubernetes. It’s always starts with just a single template: Deployment, Service, Ingress. But later on expands to >15 templates in a matter of a few days.

Oh, and starting from #Kubernetes v1.21 consider adding #OpenPolicyAgent to the list of software to manage because PodSecurityPolicy will be removed for good.

Eventually, #Kubernetes pitched like a new Ferrari, but appears to be disassembled Land Rover Defender - you have an engine you can start, but you can’t really drive it unless you assemble the rest of the car with Lego manuals in front of you.

Although #Kubernetes has no alternatives. All we need as a community to get more real life battle stories and solid tutorials beyond hello-world.

Every time I look for some resources how to develop #Kubernetes custom metric server I found myself reading that I need Prometheus and the adapter. Can we just agree that we’re doing something wrong? Can someone try and write an end-to-end walkthrough w/o saying “Prometheus”?