๐๐ฒ๐'๐ ๐ฏ๐ฒ ๐ต๐ผ๐ป๐ฒ๐๐: generally, debugging is not a fun task ๐คข
Especially for serverless, event-driven & distributed systems.
From Lambda's logging basics to ๐๐ฎ๐๐ถ๐ป๐ด ๐๐ถ๐บ๐ฒ & ๐ป๐ฒ๐ฟ๐๐ฒ๐ โ
Especially for serverless, event-driven & distributed systems.
From Lambda's logging basics to ๐๐ฎ๐๐ถ๐ป๐ด ๐๐ถ๐บ๐ฒ & ๐ป๐ฒ๐ฟ๐๐ฒ๐ โ
Lambda's a serverless technology provided by AWS.
But that ๐ฑ๐ผ๐ฒ๐๐ป'๐ ๐บ๐ฒ๐ฎ๐ป that there are no servers.
In the background, there are countless micro-containers running on top of the traditional servers.
Where do all the logs of those containers go to?
But that ๐ฑ๐ผ๐ฒ๐๐ป'๐ ๐บ๐ฒ๐ฎ๐ป that there are no servers.
In the background, there are countless micro-containers running on top of the traditional servers.
Where do all the logs of those containers go to?
By default, they will end up in CloudWatch.
Every Lambda will receive its own ๐น๐ผ๐ด ๐ด๐ฟ๐ผ๐๐ฝ.
Like a repository for logs.
Not only that, every micro-container will create a new so-called ๐น๐ผ๐ด ๐๐๐ฟ๐ฒ๐ฎ๐บ.
Think of it as a text file where logs are written to.
Every Lambda will receive its own ๐น๐ผ๐ด ๐ด๐ฟ๐ผ๐๐ฝ.
Like a repository for logs.
Not only that, every micro-container will create a new so-called ๐น๐ผ๐ด ๐๐๐ฟ๐ฒ๐ฎ๐บ.
Think of it as a text file where logs are written to.
As a micro-container can live several minutes and also serve dozens of requests in that live-time, there can be countless logs in a single stream.
If there are parallel requests, we're even having multiple log streams written in parallel!
If there are parallel requests, we're even having multiple log streams written in parallel!
๐ง๐ต๐ฒ ๐๐ต๐ฎ๐น๐น๐ฒ๐ป๐ด๐ฒ ๐๐ถ๐๐ต ๐๐ฒ๐ฏ๐๐ด๐ด๐ถ๐ป๐ด
You already see the problem: finding corresponding logs to your request in CloudWatch is cumbersome.
Even if you know the time window for your failed request, you need to browse all log streams.
You already see the problem: finding corresponding logs to your request in CloudWatch is cumbersome.
Even if you know the time window for your failed request, you need to browse all log streams.
Maybe the target log stream was created way before the failed request and contains endless other unrelated logs.
In addition, CloudWatch Console Interface is not very user-friendly. You can just load a short period of logs - scrolling up or down will load another small batch.
In addition, CloudWatch Console Interface is not very user-friendly. You can just load a short period of logs - scrolling up or down will load another small batch.
Before even starting to address the issue, you've maybe already spent 30 minutes.
So what's the solution?
We can make use of the ๐ฆ๐ง๐๐ฅ๐ง/๐๐ก๐ logs containing a dedicated request identifier (per invocation) and set up ๐๐ผ๐ด ๐๐ฟ๐ผ๐๐ฝ ๐ฆ๐๐ฏ๐๐ฐ๐ฟ๐ถ๐ฝ๐๐ถ๐ผ๐ป๐.
So what's the solution?
We can make use of the ๐ฆ๐ง๐๐ฅ๐ง/๐๐ก๐ logs containing a dedicated request identifier (per invocation) and set up ๐๐ผ๐ด ๐๐ฟ๐ผ๐๐ฝ ๐ฆ๐๐ฏ๐๐ฐ๐ฟ๐ถ๐ฝ๐๐ถ๐ผ๐ป๐.
By subscribing ๐๐ถ๐ป๐ฒ๐๐ถ๐ to our log groups and configuring a Log Parser we'll get the logs and break them up into invocations
Finally, each piece of log related to one invocation can be saved individually in a database (such as CloudSearch) for later search and debugging.
Finally, each piece of log related to one invocation can be saved individually in a database (such as CloudSearch) for later search and debugging.
You can build this yourself & try out other options like @thedashbird, where you get it out of the box ๐
Biased, as I'm dashbird's dev-๐ฅ & I'm using it for my own SaaS ๐
Also in the box:
โข well-architected hints
โข alerting (e.g. slack)
โข security warnings
... & more ๐
Biased, as I'm dashbird's dev-๐ฅ & I'm using it for my own SaaS ๐
Also in the box:
โข well-architected hints
โข alerting (e.g. slack)
โข security warnings
... & more ๐
โข โข โข
Missing some Tweet in this thread? You can try to
force a refresh
ใ
