My Authors
Read all threads
THREAD:

Is there a best instance type for Kubernetes?

It depends, but yes — some instances are better than others.
1/7

Not all CPU and memory in Kubernetes nodes can be used to run Pods.

CPU and Memory are divided into:

- Operating system
- Kubelet, CNI, CRI
- Pods
- Eviction threshold

More:
2/7

So what percentage of memory and CPU is used to run Pods?

And how does that change in cloud vendors and across instances?

Let's have a look:
3/7

Smaller instances are inefficient.

If you use a 4GB memory 1vCPU instance for your nodes, you're probably wasting 20-30% of resources.

But there's more.

There's a limit on the number of Pods you can run per Node.

- GKE 110
- AKS 250
- EKS (depends on ENIs)
4/7

In GKE, when you provision a large (256GB) node, you can't deploy more than 110 Pods.

If after 110 pods there's memory left, it's wasted.

What's the efficiency for Pods with 1GB, 2GB, 4GB, 8GB or 16GB of memory?
5/7

In EKS, the number of Pods you can have on an instance is dictated by the ENIs limits.

Again, the chart shows the efficiency for Pods with increasing memory limits.
6/7

In AKS, the chart is similar to GKE, but the graph is shifted to the right since the pod limit is higher (250 Pods).

Notice how the efficiency drops after 128GiB because you hit the Pod limits.
7/7

Summary:

- run nodes with >=64GB of memory
- at least 2vCPU
- pay attention to pod limits per node

I will open-source the research so you can run the number yourself.

Did I miss something?
What would you like to see next?

Let me know!
Missing some Tweet in this thread? You can try to force a refresh.

Enjoying this thread?

Keep Current with Daniele Polencic

Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!