Thread by @iximiuz

One of the Kubernetes superpowers is how it tackles the networking problem. Here is my approach to gaining a comprehensive understanding of the topic.

1. Learn the super-simple Kubernetes Network Model: much like VMs, every Pod gets its own IP address.

kubernetes.io/docs/concepts/…

2. Learn that achieving simplicity is hard:

2.1. Networking on a single Node. How containers communicate within a Pod, how Pods talk to each other within a Node. Long story short, it's all about Linux namespaces and network virtualization capabilities.

iximiuz.com/en/posts/conta…

2.2. Cross-node Pod-to-Pod networking. Kubernetes demands that every Pod should get its own IP. But it doesn't say how. Makes sense actually, because it's highly infra-specific. Use a plugin like Flannel or Calico instead. Keyword - overlay networks.

kubernetes.io/docs/concepts/…

3. Learn that the simplicity of the Kubernetes Networking Model means nothing for mere application developers. You cannot do much with Pod IP addresses because Pods perpetually come and go. You need a higher-level abstraction. Service to the rescue!

iximiuz.com/en/posts/servi…

3. (cont.) Kubernetes Service gives you a single IP address to access a group of Pods. It also solves the basic service discovery and load balancing problems. But there are still reliability, observability, and security problems that ideally should be tackled on the infra layer.

4. Learn that Kubernetes was build to be extended. Thanks to the Pod and Service abstractions, it's relatively simple to embed a Service Mesh layer. Every Pod gets a service proxy sidecar and all the ingress and egress traffic goes through it.

servicemesh.io

5. Until this point it was mostly about networking within a cluster. But traffic needs to get into somehow. Service can be exposed on a Node port or as a LoadBalancer but Service is an IP-layer abstraction. What's about HTTP(S)?

5. (cont.) Learn about the concept of Ingress - HTTP(S)-aware layer wiring your services with HTTP routes. As with cross-node networking, any default implementation would be opinionated (I like Nginx while someone else might like HAProxy).

kubernetes.io/docs/concepts/…

6. Much like Service Mesh is an extension of the Service layer, the Ingress layer may need to be enhanced for complex setups too. Meet API Gateways!

getambassador.io/learn/kubernet…

Cross-cutting: working with Linux networking capabilities from code is far from being a pleasant experience. Luckily, there is a lovely set of standardized executables wrapping network admin tools called CNI. Worth learning as well

github.com/containernetwo…
github.com/containernetwo…

Now, when the problem is decomposed, you just need to study one layer at a time. Below is a list of my favorite resources covering the Kubernetes networking topic (far from being comprehensive though):

Long but profound overview of Kubernetes networking.

sookocheff.com/post/kubernete…

A great series of blog posts, similar to the previous article but anyway worth reading.

Understanding Kubernetes networking:

- Pods medium.com/google-cloud/u…

- Services medium.com/google-cloud/u…

- Ingress medium.com/google-cloud/u…

Hands-on material: Container Networking From Scratch



...and its source code github.com/kristenjacobs/…

Another hands-on video: Tutorial: Communication Is Key - Understanding Kubernetes Networking

More low-level stuff: Flannel Networking Demystify

msazure.club/flannel-networ…

Another one: Cracking Kubernetes node proxy (aka kube-proxy)

arthurchiao.art/blog/cracking-…

The list is open, please share your findings as well!