๐๐๐ซ๐ฏ๐๐ซ๐ฅ๐๐ฌ๐ฌ is one of the hottest topics in cloud services. How does AWS ๐๐๐ฆ๐๐๐ work behind the scenes?
Lambda is a ๐ฌ๐๐ซ๐ฏ๐๐ซ๐ฅ๐๐ฌ๐ฌ computing service provided by Amazon Web Services (AWS), which runs functions in response to events.
๐ ๐ข๐ซ๐๐๐ซ๐๐๐ค๐๐ซ ๐๐ข๐๐ซ๐จ๐๐
Firecracker is the engine powering all of the Lambda functions [1]. It is a virtualization technology developed at Amazon and written in Rust.
The diagram below illustrates the isolation model for AWS Lambda Workers.
Lambda functions run within a sandbox, which provides a minimal Linux userland, some common libraries and utilities. It creates the Execution environment (worker) on EC2 instances.
How are lambdas initiated and invoked? There are two ways.
๐๐ฒ๐ง๐๐ก๐ซ๐จ๐ง๐จ๐ฎ๐ฌ ๐๐ฑ๐๐๐ฎ๐ญ๐ข๐จ๐ง
Step1: "The Worker Manager communicates with a Placement Service which is responsible to place a workload on a location for the given host (itโs provisioning the sandbox) and returns that to the Worker Manager" [2].
Step 2: "The Worker Manager can then call ๐๐ฏ๐ช๐ต to initialize the function for execution by downloading the Lambda package from S3 and setting up the Lambda runtime" [2]
Step 3: The Frontend Worker is now able to call ๐๐ฏ๐ท๐ฐ๐ฌ๐ฆ [2].
๐๐ฌ๐ฒ๐ง๐๐ก๐ซ๐จ๐ง๐จ๐ฎ๐ฌ ๐๐ฑ๐๐๐ฎ๐ญ๐ข๐จ๐ง
Step 1: The Application Load Balancer forwards the invocation to an available Frontend which places the event onto an internal queue(SQS).
Step 2: There is "a set of pollers assigned to this internal queue which are responsible for polling it and moving the event onto a Frontend synchronously. After itโs been placed onto the Frontend it follows the synchronous invocation call pattern which we covered earlier" [2].
Question: Can you think of any use cases for AWS Lambda?
1 - Collaboration Tools
Software development is a social activity. Learn to use collaboration tools like Jira, Confluence, Slack, MS Teams, Zoom, etc.
2 - Programming Languages
Pick and master one or two programming languages. Choose from options like Java, Python, JavaScript, C#, Go, etc.
3 - API Development
Learn the ins and outs of API Development approaches such as REST, GraphQL, and gRPC.
4 - Web Servers and Hosting
Know about web servers as well as cloud platforms like AWS, Azure, GCP, and Kubernetes
5 - Authentication and Testing
Learn how to secure your applications with authentication techniques such as JWTs, OAuth2, etc. Also, master testing techniques like TDD, E2E Testing, and Performance Testing
6 - Databases
Learn to work with relational (Postgres, MySQL, and SQLite) and non-relational databases (MongoDB, Cassandra, and Redis).
Twitter has enforced very strict rate limiting. Some people cannot even see their own tweets.
Rate limiting is a very important yet often overlooked topic. Let's use this opportunity to take a look at what it is and the most popular algorithms.
A thread.
#RateLimitExceeded
What is rate limiting? Rate limiting controls the rate at which users or services can access a resource. Here are some examples:
- A user can send a message no more than 2 per second
- One can create a maximum of 10 accounts per day from the same IP address
Fixed Window Counter
The algorithm divides the timeline into fixed-size time windows and assigns a counter for each window. Each request increments the counter by some value. Once the counter reaches the threshold, subsequent requests are blocked until the new time window begins
/1 What is the difference between โpullโ and โpushโ payments?
The diagram below shows how the pull and push payments work.
/2 ๐น When we swipe a credit/debit card at a merchant, it is a pull payment, where the money is sent from the cardholder to the merchant. The merchant pulls money from the cardholderโs account, and the cardholder approves the transaction.
/3 ๐น With Visa Direct or Mastercard Send, the push payments enable merchant, corporate, and government disbursements.
Step 1: The merchant initiates the push payment through a digital channel. It can be a mobile phone or a bank branch etc.
/2 The Netflix Engineering team selects a variety of databases to empower streaming at scale.
Relational databases: Netflix chooses MySql for billing transactions, subscriptions, taxes, etc. They use CockroachDB to support a multi-region active-active architecture.
/3 Columnar databases: Netflix primarily uses them for analytics purposes. They utilize Redshift and Druid for structured data storage, Spark and data pipeline processing, and Tableau for data visualization.