ZPOP implementation using MULTI and EXEC.

[allan bailey]

https://gist.github.com/774959

Anyone gotten a ZPOP to work for redis 2.2.2 or greater? I know there
is a plan for implementing it soon, but until then I'm trying to use
the above implementation.

Sadly, it doesn't work. I get an redis.exceptions.WatchError about
the key changing if there are any inserts/deletes from the zset.

--
Allan Bailey
zirpu...@gmail.com

[Salvatore Sanfilippo]

When the operation fails (since there was contention) you need to
reissue the operation again.
So what you need to do is to have a loop that will try again and again
until the operation is successful.

Of course if there is too much contention, there is a problem as you
need to reissue the operation too much time before succeeding. ZPOP is
not a good candidate for optimistic locking and many clients working
on the same key, as the probabilities of acquiring the lock are
actually... pessimistic ;)

ZPOP / ZREVPOP will be added to Redis unstable as soon as I or Pieter
will find the time to write an implementation (it is very easy,
actually).

Cheers,
Salvatore

>
>
> --
> Allan Bailey
> zirpu...@gmail.com
>
> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

[Pieter Noordhuis]

It can be added and there have indeed been plans. However, it can be implemented with the current command set, without using watch. Even more, you can atomically remove any range in the sorted set by simply wrapping the following commands in a MULTI/EXEC:

ZRANGE zset 0 0

ZREMRANGEBYRANK zset 0 0

The ZRANGE returns the member with the lowest score (add WITHSCORES if you also want its score), and the second command removes it. No need to call ZREM this way, as you already know what the rank of the element is you want to remove.

Similarly, popping the highest scored member:

ZRANGE zset -1 -1

ZREMRANGEBYRANK zset -1 -1

Cheers,

Pieter

[allan bailey]

On Thu, Jan 13, 2011 at 12:49 AM, Salvatore Sanfilippo
<ant...@gmail.com> wrote:
> On Thu, Jan 13, 2011 at 5:08 AM, allan bailey <zirpu...@gmail.com> wrote:
>> https://gist.github.com/774959
>>
>> Anyone gotten a ZPOP to work for redis 2.2.2 or greater?  I know there
>> is a plan for implementing it soon, but until then I'm trying to use
>> the above implementation.
>>
>> Sadly, it doesn't work.  I get an redis.exceptions.WatchError about
>> the key changing  if there are any inserts/deletes from the zset.
>
> When the operation fails (since there was contention) you need to
> reissue the operation again.
> So what you need to do is to have a loop that will try again and again
> until the operation is successful.
>
> Of course if there is too much contention, there is a problem as you
> need to reissue the operation too much time before succeeding.

Indeed, the queue is receiving too many new entries too quickly for this.

> ZPOP is
> not a good candidate for optimistic locking and many clients working
> on the same key, as the probabilities of acquiring the lock are
> actually... pessimistic ;)
>
> ZPOP / ZREVPOP will be added to Redis unstable as soon as I or Pieter
> will find the time to write an implementation (it is very easy,
> actually).

Someone has already implemented it. Check your pull requests.
It's a good place to start.

-allan

[Salvatore Sanfilippo]

On Thu, Jan 13, 2011 at 2:51 PM, Pieter Noordhuis <pcnoo...@gmail.com> wrote:
> It can be added and there have indeed been plans. However, it can be
> implemented with the current command set, without using watch. Even more,
> you can atomically remove any range in the sorted set by simply wrapping the
> following commands in a MULTI/EXEC:
> ZRANGE zset 0 0
> ZREMRANGEBYRANK zset 0 0

Oh, that's awesome Pieter! Never realized this.

So zpop is completely useless. Just two commands to model the same
thing, and there is no watch so no contention nor round trip time
wasted as MULTI/ZRANGE/ZREMRANGEBYRANK/EXEC can be sent into a single
pipeline.

I'll tweet about this, thanks!

Cheers,
Salvatore

[allan bailey]

Indeed this works fine.
I tested with multiple readers and writers w/o errors. (using redis-py 2.2.2.)

-allan

--
Allan Bailey
zirpu...@gmail.com