Lua script keys that depend on db state

[Ryan Rosenbaum]

Hello,

I am working on a basic object mapper in python to store somewhat structured data in Redis.  To "index" a field, I want to store primary keys in a set named by the value of the field.

For example, if the objects are tasks, populated with

Task(id="1", status="running")
Task(id="2", status="running")
Task(id="3", status="completed")

There would be an indexing set with key "tasks:status:running" containing "1" and "2"

Say task 2 pauses, and I want to update the status.  I would like to have a lua script to perform the following updates:

HSET task:2 status paused
SREM tasks:status:running 2
SADD tasks:status:paused 2

The issue is that because the status can be updated in different threads, I cannot ensure I have the freshest value of the status in the client, and so I cannot pipeline the above commands in the client.  Using a WATCH is also not feasible, because in practice I might update more than just the status, and I would like those updates to be applied.

I wish to make my application scalable, and so I would like to stay compatible with the Redis Cluster requirement that all keys to be updated are passed to the script.

I can implement a lock to deal with this in python, but I would like to avoid this if possible.

Does anyone have any advice for dealing with this?  I am also open to suggestions for smarter methods of indexing!


Thanks,
Ryan

[Itamar Haber]

Hello Ryan,

You need a lock, no two ways about it. A Lua script isn't a viable approach, ATM, due to cluster compatibility.

Instead of implementing your own lock, however, I recommend that you take a look at https://redis.io/topics/distlock and its existing Pythonic implementation athttps://github.com/SPSCommerce/redlock-py.

Cheers

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+unsubscribe@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

--

Itamar Haber | Technology Evangelist
Redis Labs ~/redis

Mobile: +972 (54) 567 9692
Twitter: @itamarhaber
Skype: itamar.haber

[Ryan Rosenbaum]

Hi Itamar,

Thanks for your reply.

Some part of me thought this was the case, but was hoping I'd hear otherwise.


Thanks,
Ryan

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To post to this group, send email to redi...@googlegroups.com.
Visit this group at https://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.