Grasping Kubernetes Networking (Mega Thread)

- What is Kubernetes Service?
- When to use ClusterIP, NodePort, or LoadBalancer?
- How does multi-cluster service work?
- Why both Ingress and Ingress Controller?

The answers become clear when things are explained bottom-up! 🔽
1. Low-level Kubernetes Networking Guarantees

To make Pods mimicking traditional VMs, Kubernetes defines its networking model as follows:

- Every Pod gets its own IP address
- Pods talk to other Pods directly (no visible sNAT)
- Containers in a pod communicate via localhost
2. Kubernetes does nothing for low-level networking!

It delegates the implementation to Container Runtimes and networking plugins.

A typical example: cri-o (CR) connects pods on a node to a shared Linux bridge; flannel (plugin) puts nodes into an overlay network.
3. Kubernetes Service Resource and ClusterIP

Web services normally have human-readable names. Typical web service also consists of multiple endpoints (VMs or pods).

Kubernetes Service Resource puts ephemeral pods into a named group with a ClusterIP assigned to it.
4. Kubernetes comes with its own Service Discovery

When you call a service using its domain name, KubeDNS replaces the domain name with a ClusterIP. But ClusterIP is virtual! Egress traffic to this IP is intercepted by the source node and redirected to one of the service pods.
5. Exposing Kubernetes Service (I)

Pod-to-Pod and Pod-to-Service networking work only within a cluster. From outside the cluster, it's rarely possible to access a pod by its IP.

But it's possible to map a ClusterIP to a port on every node using a Service with type NodePort.
6. Exposing Kubernetes Service (II)

No one likes services hanging on random ports. Service type LoadBalancer allows assigning a public IP address to an in-cluster service. But it must be implemented by the platform provider.

LB can send traffic to a NodePort or Pod IPs directly
7. Exposing HTTP(S) Services

NodePort and LoadBalancer services work on L4. Thus, they don't understand HTTP routing, cannot do SSL termination, etc.

An Ingress Resource describes a public L7 load balancer forwarding HTTP traffic to one or more in-cluster Kubernetes Services.
8. Kubernetes Ingress Controllers

As it usually happens with Kubernetes, it doesn't come with its own implementation of Ingress.

Kubernetes defines the Ingress Resource and expects platform providers to implement a corresponding controller to do the actual request handling.
9. Kubernetes and Service Mesh

SM transparently expands Kubernetes capabilities:

- Relies on Service resources
- Doesn't require a change on the app side
- Replaces kube-proxy for Service Discovery
- Brings multi-cluster services 🔥
- Provides Ingress Controller and/or Gateway
That's probably it for the Kubernetes networking!

For more details on a single node container networking, check out this article

iximiuz.com/en/posts/conta…
For the details of Service Discovery implementation in Kubernetes, check out this article.

iximiuz.com/en/posts/servi…
For the Service Mesh basics - the Proxy Sidecar Pattern, check out this article.

iximiuz.com/en/posts/servi…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Ivan Velichko

Ivan Velichko Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @iximiuz

30 Aug
iptables - a userspace program to configure IP packets filtration and modification rules.

It's a dated but still widely used tool:

- Linux firewall
- Container egress (SNAT) and port publishing (DNAT)
- Kubernetes service discovery
- Service Mesh transparent injection
- etc. 🔽 Image
iptables comes with its own terminology:

- tables
- chains
- rules
- targets
- policies

...that might be challenging to grasp. Here is how I approach it.

1. Come up with a logical model of packet processing inside the Linux kernel. Give meaningful names to stages. Image
2. Understand possible packet paths through the kernel.

Packet from a local process:
OUTPUT -> POSTROUTE

Packet to a local process:
PREROUTE -> INPUT

Packet forwarded between interfaces:
PREROUTE -> FORWARD -> POSTROUTE
Read 6 tweets
22 Aug
How to Expose Multiple Containers On the Same Port

First off, why you may need it:

- Load Balancing - more containers mean more capacity
- Redundancy - if one container dies, there won't be downtime
- Single Facade - run multiple apps behind one frontend

Interested? Read on!🔽
Docker doesn't support binding multiple containers to the same host port.

Instead, it suggests using an extra container with a reverse proxy like Nginx, HAProxy, or Traefik.

Here are two ways you can trick Docker and avoid adding the reverse proxy:

1. SO_REUSEPORT
2. iptables
Multiple Containers On the Same Port w/o Proxy (I)

1) Use SO_REUSEPORT sockopt for your server sockets
2) Run containers with `--network host` and the same port

SO_REUSEPORT allows binding diff processes to the same port.
--network host puts all containers on one network stack.
Read 8 tweets
21 Aug
Containers are Virtual Machines (controversial thread)

Some mental gymnastics. Bear with me.

Person A comes to Containers with prior VM experience.
Dockerfiles start FROM debian/centos/etc.
docker run/exec feels like SSH-ing sessions into servers.

Containers are VMs!
A container starts in less than a second
A VM takes tens of seconds to start

A bare-metal server can run hundreds of containers
Only a few VMs can coexist on a server

How come?

Person A starts digging into the internals to understand the difference between containers and VMs.
Person A: Aha! Containers are just isolated and restricted Linux processes + OS-level virtualization!

Person A starts sharing the finding with friends and colleagues - seasoned backend devs. Everyone instantly grasps the idea.

Then a Person B comes by. W/o prior VM experience.
Read 7 tweets
10 Aug
How to become a server-side ninja (thread)

1. Try different programming paradigms

My personal recommendation

- PHP - simplest, traditional
- Python - more generic, traditional
- JavaScript - enter the async world!
- Go - learn goroutines
- Scala/Clojure - functional
2. Try different server-side frameworks

Don't try to learn all ins and outs. Instead, learn what's common for all frameworks.

- Request handling - processes, threads, coroutines
- Request routing - how to bind code to requests attrs
- Templating
- ORM integrations
3. Learn your platform

Linux basics - sockets, i/o, filesystem
Network basics - Ethernet, IP, TCP, HTTP
DNS - hostnames are only for humans
TLS, HTTPS - how X.509 certificates work

Learn SysAdmin craft - how to install packages, configure servers, troubleshoot perf issues.
Read 5 tweets
9 Aug
The idea of Kubernetes Operators is simple and attractive.

But as it usually happens, the devil is in the details. I've been working on an operator for the past few weeks, and the learning curve is quite steep, actually.

Here are some projects that may help 🔽
1. kubernetes-sigs/kubebuilder

GitHub says it's an "SDK for building Kubernetes APIs using CRDs."

But you can scaffold an operator project with it.
2. operator-framework/operator-sdk

Much like kubebuilder, this project allows you to scaffold an operator real quick.

The difference is that it comes with batteries included:

- support of Ansible and Helm operators
- simplifies releasing operators (OLM)
- e2e testing
- linting
Read 7 tweets
7 Aug
How to grasp Containers and Docker (Mega Thread)

When I started using containers back in 2015, I thought they were tiny virtual machines with a subsecond startup time.

It was easy to follow tutorials from the Internet on how to put your Python or Node.js app into a container...
But thinking of containers as of VMs is an extremely leaking abstraction. It doesn't allow you to judge:

- what's doable and what's not
- what's idiomatic and what's not
- what's safe enough and what's not

So, I started looking for the Docker implementation details.
Docker is a behemoth doing many different things. There is plenty of materials on Docker but:

- it's either shallow and introductory container tutorials
- it's something deeply technical and hard to follow for a newbie

So, it took me a while to structure things in my head.
Read 23 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

:(