Multi key ops in Redis Cluster

[Salvatore Sanfilippo]

Hi all,

the unstalbe branch finally contains support for multiple-keys
operations in Redis Cluster.
The following is an interactive session that shows the semantics (that
will be documented in the cluster specification soon):

After the patch the obvious consequence is that we can use multi-key
ops if the keys belong to the same hash slot, and there is no
resharding in progress (the slot is said to be "stable" when it is not
in importing/migrating state):

redis 127.0.0.1:7002> set {blabla}foo 1
OK
redis 127.0.0.1:7002> set {blabla}bar 2
OK
redis 127.0.0.1:7002> mget {blabla}foo {blabla}bar
1) "1"
2) "2"

However commands with multiple keys about different hash slots are not
supported as usually:

redis 127.0.0.1:7002> mset a 1 b 2
(error) CROSSSLOT Keys in request don't hash to the same slot

Let's see what happens if the node is migrating its keys to another node:

redis 127.0.0.1:7002> CLUSTER SETSLOT 11778 migrating
34d7c5febc5883322fe3ff45e4a582b25d6d8ac5
OK
redis 127.0.0.1:7002> connect 127.0.0.1 7000
redis 127.0.0.1:7000> CLUSTER SETSLOT 11778 importing
05bebaaa67990272e4e608fa919751b2692b1ce4
OK
redis 127.0.0.1:7000> connect 127.0.0.1 7002
redis 127.0.0.1:7002>

Now the slot is set as migrating in 127.0.0.1:7002 and importing in
127.0.0.1:7000

When a slot is migrating, the node will reply to the command only if
*all* the keys mentioned exist, since if this is true, it means that
whatever is the state of the slot we have all the keys to operate in a
consistent way. The uncertainty of providing a wrong reply only
happens if at least one key is not there, the key may already be
migrated in the other node:

redis 127.0.0.1:7002> mget {blabla}foo {blabla}bar
1) "1"
2) "2"

Indeed it still works. Let's remove one of the keys (like if it was
already in the other side):

redis 127.0.0.1:7002> mget {blabla}foo {blabla}bar
(error) ASK 11778 127.0.0.1:7000

At this point the node issues an ASK redirection. We should ask to the
target node. Maybe the keys involved in the operations were already
migrated. Let's try:

redis 127.0.0.1:7002> connect 127.0.0.1 7000
redis 127.0.0.1:7000> asking
OK
redis 127.0.0.1:7000> mget {blabla}foo {blabla}bar
(error) TRYAGAIN Multiple keys request during rehashing of slot

Here we followed the -ASK protocol, connected with 7000, issued the
ASKING command, and retried the query.
The node does not have all the keys involved in the operation, since a
resharding is in progress, the only thing it can do in this case is to
reply with the TRYAGAIN error. The client should either give up and
report an error or try again after some delay.

However we can simulate that the key are already migrated just creating them:

redis 127.0.0.1:7002> connect 127.0.0.1 7000
redis 127.0.0.1:7000> asking
OK
redis 127.0.0.1:7000> set {blabla}foo 1
OK
redis 127.0.0.1:7000> asking
OK
redis 127.0.0.1:7000> set {blabla}bar 1
OK

Note how when we address a single key the "importing" node is forced
to serve our query, since we followed the -ASK protocol it is sure the
key is not in the migrating node, so can be used in the new node.

At this point the node has both the keys, it should be able to reply
to our queries involving the two keys:

redis 127.0.0.1:7000> asking
OK
redis 127.0.0.1:7000> mget {blabla}foo {blabla}bar
1) "1"
2) "1"

At this point we simulate the end of the resharding, by "closing" the slot:

redis 127.0.0.1:7000> cluster setslot 11778 node
34d7c5febc5883322fe3ff45e4a582b25d6d8ac5
OK

Now the new node will accept any request involving multiple keys, as
long as they are in the same hash slot:

redis 127.0.0.1:7000> sadd {blabla}A 1 2 3 4
(integer) 4
redis 127.0.0.1:7000> sadd {blabla}B 5 6 7 8
(integer) 4
redis 127.0.0.1:7000> sadd {blabla}C x y z
(integer) 3
redis 127.0.0.1:7000> sunion {blabla}A {blabla}B {blabla}C
1) "6"
2) "8"
3) "5"
4) "7"
5) "4"
6) "2"
7) "x"
8) "3"
9) "1"
10) "y"
11) "z"

Final considerations: with this implementation we have multiple-keys
operations in Redis Cluster, as long as the keys belong to the same
hash slot, a condition that can be forced using hash tags.

The mutliple keys operations become unavailable during the resharding
of the specific slot, unless the keys involved in the operation exist
and are already all in the source or destination node. This improves
the availability of multi key operations.

Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - GoPivotal
http://invece.org

To "attack a straw man" is to create the illusion of having refuted a
proposition by replacing it with a superficially similar yet
unequivalent proposition (the "straw man"), and to refute it
-- Wikipedia (Straw man page)

[Marc Gravell]

That's great - thanks Salvatore. I'll see if I can break it later :)

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Marc Gravell]

For info, the multi-key commands that I reported broken the other day: remain broken

https://github.com/antirez/redis/issues/1580

https://github.com/antirez/redis/issues/1581

Marc

--
Regards,

Marc

[Anirban Basu]

Hi,

 

I have a quick question.

 

We are debating to see if using redis hash will help the availability during manual resharding.

 

So if we have a redis hash with 3 fields with the same key and if we do "HMGET key field1 field2 field3", will this operation be considered as a single key operation or multi key operation as far as resharding is concerned? Or in other words will this operation be always available all the time resharding?

 

Thanks,

Anirban.

[Jan-Erik Rediger]

Hey,

the only key in `HMGET key field1 field2 field3` is `key`, so yes, this
is a single key operation and thus will only affect a single slot.

On Fri, Apr 03, 2015 at 07:05:53PM -0700, Anirban Basu wrote:
> Hi,
>
> I have a quick question.
>
> We are debating to see if using redis hash will help the availability
> during manual resharding.
>
> So if we have a redis hash with 3 fields with the same key and if we do
> "HMGET key field1 field2 field3", will this operation be considered as a
> single key operation or multi key operation as far as resharding is
> concerned? Or in other words will this operation be always available all
> the time resharding?
>
> Thanks,
> Anirban.
>
> On Friday, March 7, 2014 at 12:47:02 PM UTC-8, Marc Gravell wrote:
>
> > For info, the multi-key commands that I reported broken the other day:
> > remain broken
> >
> > https://github.com/antirez/redis/issues/1580
> > https://github.com/antirez/redis/issues/1581
> >
> > Marc
> >
> >

> > On 7 March 2014 17:53, Marc Gravell <marc.g...@gmail.com <javascript:>>

> > wrote:
> >
> >> That's great - thanks Salvatore. I'll see if I can break it later :)
> >> On 7 Mar 2014 15:20, "Salvatore Sanfilippo" <ant...@gmail.com

> >>> an email to redis-db+u...@googlegroups.com <javascript:>.

> >>> To post to this group, send email to redi...@googlegroups.com

> >>> <javascript:>.