Thread by @antirez

The Redis Cluster Proxy project received some interest and questions. Here is a small thread to inform the community about what is the plan about it, and what such new Redis component will do.

First: the proxy is designed by @artiks and myself, and developed by @artiks itself, so it means there is no need to get my hands free to go forward with this project. The same setup that made the redis-cli cluster stuff possible in the latest months basically.

What is the project *main* goal? To help with the fact that the Cluster support in clients is lacking. We want a software component that can abstract away a Redis Cluster, like if it was a single Redis instance, but with certain limitations imposed in multiple-keys commands.

So you connect to the Redis Cluster Proxy and send normal commands, without caring about redirections and hash slots or whatever. The proxy connects to the right Cluster master or slave and routes the command, and replies back to the client when possible.

However for using commands with multiple keys the user must use key tags: SUNION set:5{1111} set:6{1111}: such keys are guaranteed to be in the same host. Yet for certain commands we may provide the magic needed to abstract away the query and do multi-fetch, like for MGET.

So if we'll get MGET, Set operations, and other stuff happening magically in the proxy, it will likely be in version 2. First version just we want to abstract away the Cluster itself.

Multiplexing: normally the proxy will work like that. It has a ClusterConnections pool, and since we want to support every feature of Redis without getting mad, every client may have its private set of connections to the Cluster. However... read more:

When a new client connects, we could assign it to the GlobalClusterConnection pool to do multiplexing as long as the client does not call MULTI/EXEC, blocking commands, and so forth. As soon as it does, we create a new ClusterConnection pool for it, and continue.

This way for clients doing very simple operations without blocking operations, WAIT, transactions, and all such stuff, we are able to avoid creating a set of connections for each client, which is a huge save, without any effort. It's kinda of the same concept of copy-on-write.

The proxy project will be released under the AGPL license and will live in its own repository. We think that it will use hiredis in order to communicate with the Cluster (but the cluster logic will be inside the proxy itself).

Every client request is represented by a Request object. In this way what we do is to route a Request to a ClusterConnection object: it will send the request to the connections, and when a reply is obtained, it can send a reply object to the client.

In this way the same client may have different requests pending in different ClusterConnections at the same time, which should allow a very big level of freedom. Yet everything will be serialized from the POV of the client of course.

Many things may change in the following weeks, but that's the plan. Follow @artiks if you are interested in the evolution of the project as he may tweet progresses. Ok that's all! Cheers.

Sorry make that *replica* not slave, too used to it. Sorry but no [edit] button so far from Twitter.