Redis server-side consistent hashing

[Pallavi Agarwal]

Hello,

I was wondering why consistent hashing is always implemented at the client-side and not the server side. Wouldn't the performance be better in case of consistent hashing at the server?

[Javier Guerra Giraldez]

AFAICT, consistent hashing is mainly a criteria to choose a server from a set.

asking a server to choose a server means adding a step to every
request. it can be nice, for example client could connect any server
and then be redirected to the right one, but that makes for extra
traffic and connections before getting to the data.

--
Javier

[Greg Andrews]

I don't speak for the Redis development team, but I've been DevOps at a company who rolled our own key/value store which implemented server-side sharding.  Here are the benefits and drawbacks as I see them:

Server-side sharding means the client does not know which of the servers has the desired key.  There are generally two approaches to handle this:

• The client connects to a server and issues the GET/SET command.  The server looks up the key in the hashing algorithm and returns a redirect to the server that has the key.  The client connects to the new server and re-issues the GET/SET command, and receives the answer.
• The client connects to a server and issues the GET/SET command.  The server looks up the key in the hashing algorithm and makes a proxy connection to the server with the key, passes the client's command to the other server, passes the answer back to the client, and disconnects from the other server.

Obviously, if the client happens to be connected to the server with the key, the server will return the answer directly.  However, the more shards you have, the smaller the percentage of time this will be true.

The benefits to the client code are obvious:  The client code and configuration is extremely simple.  The client doesn't need to implement a sharding algorithm, and only needs to know about one of the Redis servers.

The drawbacks to the server code are the same:  The server code is more complex, implementing the sharding algorithm and the redirect answer or the proxy routines.  Obviously the proxy routines increase the server complexity more than the redirect.

But there's a performance penalty for both approaches:  For a large percentage of cases, the GET/SET command must be sent twice before the client receives the authoritative answer from the correct server.  Performance can be worse with server-side sharding rather than better.

There are some other tradeoffs, but the above is the main one.

  -Greg

On Fri, May 15, 2015 at 2:19 AM, Pallavi Agarwal <p1ag...@eng.ucsd.edu> wrote:

Hello,

I was wondering why consistent hashing is always implemented at the client-side and not the server side. Wouldn't the performance be better in case of consistent hashing at the server?

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[The Baldguy]

In regular standalone Redis it isn't implemented in the server because it doesn't make sense. You would be hashing all slots to the same server. Redis is a single-server setup so running a shard algorithm to map all keys to one server is a waste and pointless.

However, in Redis Cluster, which is designed to be in a cluster, shard selection is done on the server side with the client needing to redirect to the correct server and cache the topology as it discovers it.