updating zset score from multiple clients

[Raghava Mutharaju]

Hello all,

I have the following scenario:

Redis server maintains several zsets and elements are added to these zsets by multiple clients connected to this server. The score needs to be an increment of the previous highest score of that particular zset -- all the new elements that are added to a zset at a particular point of time could have the same score but the score needs to be a value greater than the previous highest score of that zset.

How can multiple clients assign scores to their insertions such that the scores of the zset are in increasing order?

Note: 

1) Having all the clients maintain the current highest score of the zset does not work since it could be changed my the addition of elements by another client to that zset.

2) I can perform 2 operations -- read highest score and then do a zadd. I want to avoid 2 operations for this and see if it can be done with a single call.

3) The java client, Jedis, does not yet support scripting. 

My current solution is to have a proxy server which first intercepts all the zadd commands. This proxy maintains the latest score and performs the zadd with the incremented score (since all the zadds go through this, it works). But this is not an elegant solution + if there are multiple servers then I need to run this proxy server on all of those redis servers. 

I guess the above scenario is pretty common and there should be better solutions to it. What are the other possible ways in which this could be done?

Thank you in advance.

Regards,

Raghava.

[Josiah Carlson]

On Wed, Jan 25, 2012 at 6:18 AM, Raghava Mutharaju
<m.vijay...@gmail.com> wrote:
> Hello all,
>
> I have the following scenario:
>
> Redis server maintains several zsets and elements are added to these zsets
> by multiple clients connected to this server. The score needs to be an
> increment of the previous highest score of that particular zset -- all the
> new elements that are added to a zset at a particular point of time could
> have the same score but the score needs to be a value greater than the
> previous highest score of that zset.
>
> How can multiple clients assign scores to their insertions such that the
> scores of the zset are in increasing order?

If scripting was available in Jedis, I would say use that. But since
you mention it is not, you have 2 options available to regular Redis:

1. WATCH/MULTI/EXEC - WATCH your zset, fetch the value, MULTI, ZADD
your value, EXEC. If it fails, you retry. If you have high contention,
this will be very slow as you may be retrying more than you are
adding.
2. Use a lock. More specifically, translate this into Java and use
that: http://dr-josiah.blogspot.com/2012/01/creating-lock-with-redis.html
. You acquire the lock, perform your operation, then release the lock.
For high-contention objects, using a lock can actually improve
performance significantly.

Note that without scripting, your best-case scenario is 2 round trips
(assuming no concurrent readers, which is not the case for you), 3
round trips if you use WATCH/MULTI/EXEC (more if there is contention),
and at least 6 if you use the lock I linked to (2 to acquire in the
best-case, 2 to release in the best-case, and 2 to actually do your
operation; but retries compared to WATCH/MULTI/EXEC are eliminated).

> Note:
>
> 1) Having all the clients maintain the current highest score of the zset
> does not work since it could be changed my the addition of elements by
> another client to that zset.
> 2) I can perform 2 operations -- read highest score and then do a zadd. I
> want to avoid 2 operations for this and see if it can be done with a single
> call.
> 3) The java client, Jedis, does not yet support scripting.

> My current solution is to have a proxy server which first intercepts all the
> zadd commands. This proxy maintains the latest score and performs the zadd
> with the incremented score (since all the zadds go through this, it works).
> But this is not an elegant solution + if there are multiple servers then I
> need to run this proxy server on all of those redis servers.
>
> I guess the above scenario is pretty common and there should be better
> solutions to it. What are the other possible ways in which this could be
> done?

Your particular use-case isn't terribly common, but needing to update
existing data is fairly common. Scripting, WATCH/MULTI/EXEC, and locks
are the typical solutions.

If you feel ambitious; you could write some C, update Redis to have
the command that you want, and update Jedis to support it. That may
ultimately be the right solution, even though it may be quite a bit of
work.

Regards,
- Josiah

[Raghava Mutharaju]

Thank you Josiah :). I will check out the options.

Regards,

Raghava.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To post to this group, send email to redi...@googlegroups.com.
To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.