variadic EXISTS

[Xiangrong Fang]

Why the EXISTS command is not variadic? Is there any plan to support that?

Thanks,
Shannon

--

[Pieter Noordhuis]

The commands that have been made variadic were made variadic on the value's members, not on keys. Since a single cannot be distributed over multiple machines, it is safe to assume that all list elements, set members, sorted set members and hash fields are on the same machine. For a single command accepting multiple key arguments, this is not the case.

There are some legacy commands that can take more than 1 key as input, but these are generally hard to distribute. For instance, consider SORT where the data set is spread out over two or more machines. When it gets called with a GET argument, there is no way to tell upfront which keys will be located on which machine. Because commands whose output depend on more than 1  key cannot be supported out of the box, it is unlikely that new commands of this kind are added, or existing commands will be modified to depend on more than 1 key.

With 2.6, testing existence of multiple keys on the same machine can be trivially added using a Lua script. But, because we can't support the generic case it will not be added natively.

Cheers,

Pieter

--

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

[Marc Byrd]

Before Pieter's sage (as always) reply, I was going to say (in my pre-cluster thinking):

Simply put it in a multi-exec, which has the same effect.   Atomic multi-exists.  (And btw variadic exists is quite an existential question!).  

But Pieter's reply took a different direction, which leads me to wonder:  Which way will redis go?  Is atomic more important, or is cluster?  How does redis achieve both?  

More specifically (and I do apologize if I've missed a reading assignment):  how does clustering redis handle multi-exec (much less scripts) when the keys involved live on different servers?

Best,

Marc

[Josiah Carlson]

On Tue, Dec 13, 2011 at 9:45 PM, Marc Byrd <dr.mar...@gmail.com> wrote:

Before Pieter's sage (as always) reply, I was going to say (in my pre-cluster thinking):

Simply put it in a multi-exec, which has the same effect.   Atomic multi-exists.  (And btw variadic exists is quite an existential question!).  

But Pieter's reply took a different direction, which leads me to wonder:  Which way will redis go?  Is atomic more important, or is cluster?  How does redis achieve both?  

More specifically (and I do apologize if I've missed a reading assignment):  how does clustering redis handle multi-exec (much less scripts) when the keys involved live on different servers?

It raises an error, as it is not supported.

You can choose the bucket and machine that a key is on (via the use of keypart{part_to_be_hashed_for_bucket}otherkeypart ), so you can keep associated data together for those multi-key operations.

Also, from what I understand, Redis will support the manual migration of hash buckets, so you can migrate an entire bucket of keys and operate on your multi-key stuff as necessary. Probably frowned upon, but still possible. 

Regards,

 - Josiah

[Xiangrong Fang]

Actually while I ask this question I am considering about performance and simplicity, but not atomic.   Pieter's reply is very clear about why it does not suit well with cluster. So I will just use a pipeline.

2011/12/14 Marc Byrd <dr.mar...@gmail.com>

--

[Salvatore Sanfilippo]

The problem is not "just" how it works in the cluster, since after all
we stated it in our Redis Manifesto: we accept having two APIs, one
works only in the single instance, one (a subset) works in the
cluster, so if something is very, very useful, like set intersections
are, it's worth to have in the API even if we'll not support it in the
cluster.

But at the same time, it I completely agree with Pieter that when
adding a command this can't be just a detail... if it is going to work
well in the distributed version without very big advantages, the
friction to enter the API will be much larger.

1) So what are the additional problems? That EXISTS can't be turned
into variadic without changing the return type from integer reply to
multi bulk reply.
2) Even if we accept breaking the API with 3.0 and always return a
multi bulk I don't like that a command that is often useful against a
single key will return an array from the client point of view.
3) So this would require a new command, MEXISTS... but we are entering
the scripting era... in very short time and this will not be an issue
and will be up to the user to write a short script.

Btw here is the script:

local retval,j
retval={}
for i=1,#KEYS do
table.insert(retval,redis.call('exists',KEYS[i]))
end
return retval

./redis-cli --eval /tmp/mexists.lua foo bar key0
1) (integer) 0
2) (integer) 0
3) (integer) 1

Cheers,
Salvatore

On Wed, Dec 14, 2011 at 10:05 AM, Xiangrong Fang <xrf...@gmail.com> wrote:
>
> Actually while I ask this question I am considering about performance and simplicity, but not atomic.   Pieter's reply is very clear about why it does not suit well with cluster. So I will just use a pipeline.
>
>
> 2011/12/14 Marc Byrd <dr.mar...@gmail.com>
>>
>> Before Pieter's sage (as always) reply, I was going to say (in my pre-cluster thinking):
>>
>> Simply put it in a multi-exec, which has the same effect.   Atomic multi-exists.  (And btw variadic exists is quite an existential question!).
>>
>> But Pieter's reply took a different direction, which leads me to wonder:  Which way will redis go?  Is atomic more important, or is cluster?  How does redis achieve both?
>>
>> More specifically (and I do apologize if I've missed a reading assignment):  how does clustering redis handle multi-exec (much less scripts) when the keys involved live on different servers?
>>
>> Best,
>>
>>
>> Marc
>>
>>
>> On Tue, Dec 13, 2011 at 9:30 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
>>>
>>> The commands that have been made variadic were made variadic on the value's members, not on keys. Since a single cannot be distributed over multiple machines, it is safe to assume that all list elements, set members, sorted set members and hash fields are on the same machine. For a single command accepting multiple key arguments, this is not the case.
>>>
>>> There are some legacy commands that can take more than 1 key as input, but these are generally hard to distribute. For instance, consider SORT where the data set is spread out over two or more machines. When it gets called with a GET argument, there is no way to tell upfront which keys will be located on which machine. Because commands whose output depend on more than 1  key cannot be supported out of the box, it is unlikely that new commands of this kind are added, or existing commands will be modified to depend on more than 1 key.
>>>
>>> With 2.6, testing existence of multiple keys on the same machine can be trivially added using a Lua script. But, because we can't support the generic case it will not be added natively.
>>>
>>> Cheers,
>>> Pieter
>>>
>>> On Tue, Dec 13, 2011 at 6:32 PM, Xiangrong Fang <xrf...@gmail.com> wrote:
>>>>
>>>> Why the EXISTS command is not variadic? Is there any plan to support that?
>>>>
>>>> Thanks,
>>>> Shannon
>>>>
>>>> --
>>>>
>>>>

>>>> --
>>>> You received this message because you are subscribed to the Google Groups "Redis DB" group.
>>>> To post to this group, send email to redi...@googlegroups.com.
>>>> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
>>>> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Redis DB" group.
>>> To post to this group, send email to redi...@googlegroups.com.
>>> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
>>> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Redis DB" group.
>> To post to this group, send email to redi...@googlegroups.com.
>> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>
>
>
> --
>
>
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele