Thread by @oliverjumpertz

I love AWS Lambda. It's so easy to get a function up and running while AWS handles everything other than the code for you.

But where is your code actually executed? How does AWS do it?

Well, I took a deeper look and this is what Lambda's foundation is!

🧵👇🏻

1️⃣ Thinking about Lambda

We know that we can write a script that looks like below and then upload it to AWS while they take care of everything else.

And from simply looking at the code, it doesn't look like the usual API method we'd implement in Express.

We're exporting a function, so something else must take our code, import it, and handle everything else. 🤔

Conclusion number one:
Something must run our code.

We also know the infamous cold start issues, which became better over time but are still there. Sometimes the environment is shut down and then started again.

Conclusion number two:
Whatever runs our code can be shut down and started again.

Have you ever noticed that it's actually impossible to access anything on the host system other than the temp directory? If not, try to, and you'll see that the environment will prevent it.

Conclusion number three:
The environment is pretty secure.

2️⃣ Thinking About The Technology Lambda Is Based On

There are a few ways of how AWS could have implemented Lambda (taking into account its initial release year of 2014):

- Containerization
- Virtualization
- Something running on bare metal

We can quickly rule out "Something running on bare metal." AWS already had EC2 at that time and some good knowledge of virtualization.

It would not have made a lot of sense for AWS to step back from virtualization and not use their existing infrastructure.

They basically had everything in place to provision a virtual machine on the fly.

What about Containers, then?
They can be spun up quickly and disposed of again.

AWS could have taken the code, wrap it with something and then put it inside a container.

This would have been a great idea but also something completely new for AWS at that time.

Additionally, it would not explain the (old) cold start issues because containers are usually pretty fast to spin up.

What about virtualization, then?
It would make a lot of sense.

At the time of starting Lambda, AWS already had EC2 and all the infrastructure to provision a virtual machine on the fly. It would also explain why a lambda function being cold-started could sometimes take ...

... so long until it finally served a request. But how did they manage to reduce the cold start time?

Before we dive deeper, I'll give you the answer:
Lambda has, since its release, been based on virtualization technology. No fancy containers, nothing self-developed.

It simply made the most sense for AWS to do it exactly this way.

As you've read above, they had all the knowledge, and they had the infrastructure for provision.

All they had to add was something to wrap user functions and something to call them, as well as some ...

... supporting services which could handle eventing.

And now that we know that it's virtualization, we can look at what's exactly used nowadays!

3️⃣ Enter Firecracker

Firecracker is a virtualization technology, or better, a virtual machine monitor (VMM) developed at Amazon (now open-sourced) and written in Rust (🧡).

It's the engine powering all your Lambda functions.

firecracker-microvm.github.io

What Firecracker basically does is creating and managing a multitude of Linux Kernel-based Virtual Machines (KVMs), which are microVMs that are faster and more secure than traditional VMs.

The interesting thing about those microVMs is that they are actually on par with...

...containers regarding memory footprint and start-up time, all while offering even more security due to the advanced features KVM offers.

You can read more about KVM at the link below.

en.wikipedia.org/wiki/Kernel-ba…

Firecracker comes with a REST API used to create VMs, delete them, manage them, etc.

Whenever you create a new lambda function and upload your code, the Firecracker REST-API is called under the hood to create a microVM with your function's CPU and memory settings.

AWS keeps base images that contain language/runtime-specific bootstrap code.
This is the code that actually invokes your handler, passes it the request, and takes the response to return it to the caller.

And it is also the code where various metrics are measured...

...that are then used to calculate your bill.

You can imagine the code as containing an infinite loop, waiting for requests, passing them to your function, returning the response, and gathering execution metrics.

After Firecracker created a new microVM, including your language-specific runtime, your code is put into its /var/runtime/bin folder. This is the place where the bootstrap code resides, too.

Now your function is basically able to run and accept requests.

After a while, AWS will shut the VM down to save resources on their side.

This is, once again, a call to the Firecracker API.

Incoming requests, e.g., through API Gateway, lead to Firecracker being tasked to start the VM again, such that it can process the request.

And that's it so far for Firecracker and its job within AWS Lambda.

4️⃣ Surrounding Infrastructure And Supporting Services

There are, of course, many surrounding systems and services that do their work to make AWS Lambda what it is.

There are services and systems around Firecracker that make all those requests to its API.

Some services are routing the requests. Other services decide when to call Firecracker to shut a certain VM down or pause it and when to spin it up again.

And there are certainly a lot more services, like Queues, scheduling asynchronous messages, and much more.

5️⃣ A Conclusion

Especially the improvements of cold-starts within recent months and years made me initially guess that AWS made the switch from virtualization to containerization.

But as you see, I was wrong and stand corrected. I would not have guessed that using KVM was an appropriate answer to tackle those issues.

It's pretty interesting to see how AWS has gone to make Lambda what it is today, and even more interesting to see Firecracker and how it solves many problems serverless functions cause for service providers.

And that's it. This thread's over.

If you like threads like this one, a follow would mean the world to me! 💛🙏🏼

I try to create them regularly so that you can learn!