Logical isolation is as important as physical isolation when it comes to fault isolation.
Unless you’re serving identical requests arriving at an identical rate, requiring identical amount of work in a completely static environment, every server process is a multi tenant system.
Unless you’re serving identical requests arriving at an identical rate, requiring identical amount of work in a completely static environment, every server process is a multi tenant system.
It’s important when designing such services to be able to protect against misbehaving *class* of requests that trigger some form of degraded of pathological behavior, potentially affecting the response time of all other requests.
I think such services should be able to classify the requests they receive in some form (by assigning a QoS, or by identifying the client, or identifying a request profile) and be able to shed load *proactively*.
“Constant work” systems need to be shedding load constantly too.
“Constant work” systems need to be shedding load constantly too.
Most talk about backpressure treats all requests as mostly identical
I’m thinking of something like a per request-class backpressure. Where requests that disrupt the steady state (SLO) are penalized (outright rejected or put in low priority queues) more than requests that don’t.
I’m thinking of something like a per request-class backpressure. Where requests that disrupt the steady state (SLO) are penalized (outright rejected or put in low priority queues) more than requests that don’t.
Some implementations achieve this by having multiple in-process queues.
Where different “types” of requests go into different queues, with the highest priority queues being processed (often in a LIFO manner), and lower priority requests being queued.
Where different “types” of requests go into different queues, with the highest priority queues being processed (often in a LIFO manner), and lower priority requests being queued.
When in-process queues are full, requests are outright rejected.
With enough client-side jitter when retrying, and an “upstream” aware load balancer that knows not to send a client request to a previously rejected backend, this can work fairly well?
Anyone know of other ways?
With enough client-side jitter when retrying, and an “upstream” aware load balancer that knows not to send a client request to a previously rejected backend, this can work fairly well?
Anyone know of other ways?
