Do you really need a dedicated message queue? Twitter thread edition! Let's go 1/11
So notado.app is pretty simple, right? It just saves text. But sometimes that same text needs to be forwarded to @instapaper , @Pinboard or @readwise. Other times, users import a buttload of saved stuff from @Twitter , @Reddit , #HN or @AmazonKindle 2/11
If you were to always wait for all of this extra stuff to finish before responding to a user, the UX would be crappy and slow 3/11
This is where many apps use message queues like #sqs, #rabbitmq and #kafka - they send a description of what needs to be done for another process to do later 4/11
The downside is that you now have more things to deploy and manage or pay for (as managed services) - for the non-FAANG apps dealing with small payloads of text data, it can sometimes to be possible to avoid all of that extra overhead 5/11
Sergio wrote a great comment on the #Rocket repo questioning whether a user really needed a dedicated job/task system with #tokio and async #Rust: github.com/SergioBenitez/… 6/11
notado.app took this idea and ran with it; any long-running" operations (in practice, never more than 1-2s) in HTTP handlers are handled by a task spawned by #tokio, leaving HTTP handlers to respond pretty much immediately 7/11
There are a whole bunch of ways that you can refine this further, but done right, this approach can take you very far as a small (ie. not FAANG-scale) app before you need to consider bringing in a dedicated message queue 8/11
My mental model has been so warped by working at scale that my default instinct is always to reach for what I know, and what I know is usually completely overkill for my modest needs as a solo dev 9/11
It’s easy to fall into the trap of doing a little with a lot, so here's to finding more ways to do a lot with a little 10/11
Wanna read a more comprehensive version of this thread? There's an article about it here notado.substack.com/p/do-you-reall… 11/11
• • •
Missing some Tweet in this thread? You can try to
force a refresh
