I see devs getting frustrated at timeouts like they're the issue.
Timeouts are not generally the problem. They are safeguards *against* the problem. They are there to prevent ever-escalating pile-up of cascading resource exhaustion and provider a self-recovery mechanism.
Timeouts are not generally the problem. They are safeguards *against* the problem. They are there to prevent ever-escalating pile-up of cascading resource exhaustion and provider a self-recovery mechanism.
I'm going to use Stack Overflow as an example, but this fits every app.
We render pages in ~18-20ms usually. Great. But what happens when something goes slow? What if SQL takes 2,000ms all of the sudden? What if Redis takes 2,000ms? That's a HUGE increase. When this happens...
We render pages in ~18-20ms usually. Great. But what happens when something goes slow? What if SQL takes 2,000ms all of the sudden? What if Redis takes 2,000ms? That's a HUGE increase. When this happens...
When normally taken-for-granted fast things go slow, everything waiting on it piles up. Pipelines, connection pools, thread pools, server request queues, load balancers, network stacks...it's a massive snowstorm freeway accident that you can't avoid. Everything falls over.
Timeouts are a cap on the damage/impact. Those victims are written off for the greater good. If we wait on them forever, the pile up becomes too great to recover. Our resources throughout the stack are exhausted and practically users may wait seconds or minutes to get a page.
When servers become unresponsive because processors are spending all of their time *waiting*, context switching, checking thread queues, etc. the game's over if something doesn't give.
Timeouts are that "give".
They are the moving of that accident to the side of the road.
Timeouts are that "give".
They are the moving of that accident to the side of the road.
Don't think of timeouts as your enemy. They're not. If your timeouts are too sensitive, yeah sure, adjust them. But they are your friend.
The best way to realize this is to think it through and consider the alternative: what would happen if the timeouts *weren't* there?
The best way to realize this is to think it through and consider the alternative: what would happen if the timeouts *weren't* there?
