,
9 tweets,
2 min read
Read on Twitter
Queues are bad, but software developers love them. You'd think they would magically fix any overload or failure problem. But they don't, and bring with them a bunch of their own problems.
First off, queues turn your system into a liar. Convert something to an async operation and your system will always return a success response. But there's no guarantee that the request will actually ever be processed successfully.
They also break consistency. Under normal operations, a queue might drain write operations quick enough to give the impression of read-your-writes consistency. In the concurrent case, a backlog results in reordering, with older data overwriting newer data.
Oh but what about ORDERED queues? The only way to get ordered application of writes is to perform them one after the other. That means the queue must be processed 1 item at a time. If it takes 10ms to process a request, that's only 100 req/s.
In some cases the queue can be partitioned by key or what-not, with dedicated consumers per queue. So now we find ourselves in this very complex world. How are blocking failures handled? In the distributed case, how do we expand capacity without disruption? What about hot keys?
On the ops side, queues give the illusion of capacity. They can certainly smooth out short spikes, but they're not going to make your database suddenly able to process more requests. They have the nasty tendency to mask bottlenecks until too late.
They're also hard to capacity plan themselves. Queues use a small amount of RAM/disk during normal operation. Suddenly utilization goes from near-zero to a many order of magnitude increase. These sorts of instant modality shifts are notoriously difficult to prepare for.
So what to do? In almost all cases it's better to propagate failure and backpressure. This gives a chance for the upstream to decide if it wants to retry or fail itself. It keeps things simple and easy-to-understand. It doesn't build up huge backlogs that become disasters.
All said, that's not to say that there aren't legitimate use cases for queues. A huge dose of skepticism should be applied. There's no such thing as a good queue, only one that is absolutely necessary.
