Messaging Queues are one of the critical components in building distributed & scalable applications.

Let's talk about them in this thread:

- what is it ?
- how to use it ?
- benefits of using it in an architecture.


As you might already know a queue is a data structure which works in first-in first-out manner.

It basically contains a sequence of objects that can be read and processed in the same order in which they were received.


A message is any data which can be in different formats like text, json, xml etc depending your application needs.

It contains all the information that needs to be sent from sender application to the receiver application.

For example: If we're building a system to send emails asynchronously, this is how a message looks like:

"email_from": "",
"email_to": "",
"subject": "Test Message",
"mail": "This is test email body."

Thus, a messaging queue provides an asynchronous protocol between sender and receiver applications.

Sender application send msgs to the queue and does not wait for any immediate response. Receiver application connects to this queue and receives and processes the messages.

Decoupling and Scalability:

👇🏻 Image
A decoupled system is achieved when two or more systems can work independently without being connected to each other.

Separating out and decoupling systems like this helps us to maintain and scale them easily.

If the receiver applications is down, the sender application can still continue to serve the requests and push messages in to the queue.

Once the receiver application comes back up again, it can pickup and process the pending messages.

Advantages of a decoupled system:

- can evolve independently
- can be scaled independently
- can be written in different languages
- can be managed by different teams

Let's discuss a use case:

Imagine we have a service which receives thousands of requests per second to generate invoice pdfs for your customers.

In this case we can have a main API service which handles the requests,a queue and a worker service which processes the messages.

In this case the API service doesn't need to have the logic of all the calculations and generating the PDFs. It would just insert a message into the queue for every invoice generation request and return a response.

Worker service then picks up the messages and generates invoices for individual requests and then send the invoice PDFs to your customers.

In future if we want to scale the application to generate more invoices per second, we can just add more workers to the worker service.

Even if the processing of a particular message fails, we can add retry logic and delete the message from queue only when it is processed successfully.

If you want to learn cloud computing concepts like these which are very important to know to designing systems, I recommend buying @dvassallo's AWS course:…

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Sunil Kumar

Sunil Kumar Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @sunilc_

5 Nov
Data Structures and Algorithms articles for beginners:

🧵👇🏻 Image
Grokking Algorithms — Book Review…
How to prepare for competitive programming ?…
Read 11 tweets
3 Nov
Top 10 Python libraries for data science:

1. Pandas

Pandas is an open-source Python package that provides high-performance, easy-to-use data structures and data analysis tools for the labeled data in Python programming language.
2. NumPy

One of the most fundamental packages in Python, NumPy is a general-purpose array-processing package. It provides high-performance multidimensional array objects and tools to work with the arrays. NumPy is an efficient container of generic multi-dimensional data.
Read 12 tweets
2 Nov
A list of programming languages that are actively developed on GitHub:

1. GoLang ( ⭐️78.1k )
2. TypeScript ( ⭐️65.3k )…
Read 11 tweets
30 Oct
Micro Services Architecture Vs Monolith Architecture:

You might have heard about these terms again and again. But what are they, their advantages and disadvantages?

Let's discuss it in this thread: 🧵👇🏻
Micro Services Architecture:

Micro Services is an architectural style where a single application is built as a suite of multiple smaller services which are loosely coupled, maintainable and scalable separately.

Let's discuss what that means.

Let's take the below example:

As you can see, our example application takes care of user accounts, reading and updating inventory and shipping the products to end users.

Read 16 tweets
29 Oct
DATABASE CONNECTION POOLING is one of the import concepts to know for every Software Developer (both front-end and back-end).

You should know how REST services are optimised using database connection pooling.

Let's discuss it in this thread 🧵👇🏻
Let's say you have a backend service which has an API endpoint to read data from a database table.

This is how a naive developer would implement such an endpoint:

- open db connection
- read the data from database
- close db the connection
- return the response

Now there's nothing wrong with implementing it this way. The end point works perfectly fine and it returns the desired results.

The problem is when you get so many concurrent requests.

Every API call is going to open a database connection and that's not good because...

Read 10 tweets
27 Oct
Are you a freelancer and struggling to manage your time between different projects & clients?

Here's a list of tools and resources to help you organise your work:

1. Beagle: A tool for creating better contract proposals
2. Funnel: A CRM for small businesses and freelancers
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!