Re: Redis "smart client" with internal routing table

[Salvatore Sanfilippo]

On Fri, Mar 15, 2013 at 12:09 AM, Phil Kallos <phil....@gmail.com> wrote:
> I did some preliminary searching on the net and this group looking for
> projects that do this; I didn't find any so my first question is: do any
> such projects already exist?


Hello Phil!

There is Predis with some Redis Cluster support, and I'm writing a
client myself that wraps redis-rb:

https://github.com/antirez/redis-rb-cluster

Basically how to write a proper Cluster client is a work in progress
as Redis Cluster itself is a work in progress, however I'm trying to
write redis-rb-cluster so that it is very easy to understand and
containing what I think are the best practices for a Cluster client.

There are many "smart" things that could be added to a Cluster client
to improve parallelism given that we have many nodes, for now I'm not
implementing it, but just trying to provide a viable solid client to
talk with a Redis Cluster.

In the long run the hope is to have a good Redis Cluster client for
every used programming language, but my guess is that it makes a lot
more sense to write a Cluster client as a *wrapper* to an existing
standard client instead of rewriting it from scratch.

Cheers,
Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter

[Josiah Carlson]

For a lot of users who are sharding already, their client already has
a method of hooking in a "shard hint" to determine what shard to
connect to, which then creates/re-uses a connection to the right
server. The only thing that gets interesting about this with Redis
cluster is that shard assignment can change during runtime. Most of
the time it shouldn't be an issue, as the server you were connecting
to for shard X will be up, so even if that shard moved servers, you'll
get the notification, which you can use to update local configuration.
Where it gets interesting is when that Redis cluster node goes down,
as then the client will have to ping a different server to find out
the updated configuration of the cluster.

I haven't read the docs lately, so I don't know how long it is
expected to take for a cluster node being down to have its shards
migrated to one of the slave replicas. But that will limit how quickly
clients can recover from a cluster node failure.

- Josiah

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Phil Kallos]

In the long run the hope is to have a good Redis Cluster client for 
every used programming language, but my guess is that it makes a lot 
more sense to write a Cluster client as a *wrapper* to an existing 
standard client instead of rewriting it from scratch. 

I agree this is a great model to strive for. I looked at your redis-db-cluster lib for ruby and that's exactly what I wanted! I will try to contribute and for fun write a golang library to accomplish the same. I'm certain I'll have more questions as I get started!!

Thanks very much for taking the time to answer. 

[Salvatore Sanfilippo]

On Fri, Mar 15, 2013 at 3:51 AM, Josiah Carlson
<josiah....@gmail.com> wrote:
> For a lot of users who are sharding already, their client already has
> a method of hooking in a "shard hint" to determine what shard to

Exactly, this is already a common practice after all. The difference
with Redis Cluster is mainly that:
1) As you mentioned, slots can be moved, so the ability to update
config is needed.
2) That you have the CLUSTER NODES command to simply update the state
after a cold start. Actually in my client I always use CLUSTER NODES
once I detect that a single route changed: it is much more efficient
in case multiple slots moved after a resharding.
3) You have to handle the ASKING thing (if you are redirected with
"-ASK" instead of "-MOVED" you have to issue an ASKING command to tell
the server you are here because of a redirection).

Well nothing exceptional or too complex after all, however as usually
the evil is in the details.
For instance in redis-rb-cluster you can also configure the max number
of sockets you want to use.
Another subtleness is that it is important to always take a list of
random IPs of nodes so that you have somebody to connect to if the
node you are connecting is failing.

> I haven't read the docs lately, so I don't know how long it is
> expected to take for a cluster node being down to have its shards
> migrated to one of the slave replicas. But that will limit how quickly
> clients can recover from a cluster node failure.

This is configurable and is called the "node timeout", all the other
timings in the cluster are multiple of this value.
For instance, max time a slave does no longer receive updates from
master to still be elegible, is node_timeout * 10.
Time needed for a node to be marked as ok again if it returns to be
reachable via cluster bus, and nobody failed over it for some reason,
is node_timeout * 10, and so forth.

It is possible that even the multiplication factors may turn into
configurable parameters later, but I would do it only if absolutely
needed as it is more stuff to understand from the POV of users.

[Salvatore Sanfilippo]

Thank you, in the meantime there are a lot of progresses on Cluster.
I plan to make a screencast the next week as some kind of howto about
how to play with cluster and to show things like cluster creation,
resharding, adding a new node, high availability and so forth.

Salvatore

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Phil Kallos]

So something occurred to me: for the sake of a web application without a persistent memory model across requests, it would seem useful to have a cluster client that maintains a cached map of "key"->"cluster node".

My first instinct would be to make a persistent daemon service to handle the persistence, then have the web application connect over the network loopback -- but that approach seems really goofy. I know if we want to be implementation-specific I could make a PHP extension that persists the "key"->"cluster node" map, but what's a more general solution?

For the sake of a web application or web service connected to a "smart redis cluster client" that maintains state, what is the right model?