How do do a round robin to slaves using redis-py and a lua script

[David Montgomery]

Hi,

I quite don't know best practice for reading from multiple slaves on
multiple ports. I am doing poor mans load balancing. I use route 53
and a have a slave domain where I register the ip address of the
slaves. I dont have the luxury of HA proxy at the moment.

On each slave I have two cores and two running instances on 6379 and
6380. I use choice to randomly select the port to read and depend on
route 53 for selecting the slave ip.

from random import choice
r_main_reads = [redis.StrictRedis(host='slave.com',socket_timeout=.09,port=6379),redis.StrictRedis(host='slave.com',socket_timeout=.09,port=6380)]
choice(r_main_reads).get('foo')


A issue I encountered was trying to load a lua script and to execute
as per the below

cookie_match = """
local ckid = redis.pcall('hget',KEYS[1],ARGV[1])
local meta = redis.call('hgetall', ckid)
return {ckid, meta}"""
get_cookie = choice(r_main_reads).register_script(cookie_match)
def execute_get_cookie(guid):
data = get_cookie(keys=['my_user_id'], args=[guid],
client=choice(r_main_reads))
return data


So...to test...I have 4 terminals open. One for each slave and port.
xxx.xxx.xxx.xxxx:6379 xxx.xxx.xxx.xxxx:6380 yyy.yyy.yyy.yyy:6379
yyy.yyy.yyy.yyy:6380. As a test..I did a write and all slaves and
ports were updated.

When I load the script and I execute in the main io loop only the
below servers are hit

xxx.xxx.xxx.xxxx:6379 yyy.yyy.yyy.yyy:6380

From what I gather is a socket is opened to one port on one of the
servers. Two use all I would image I would have to use ip address
which I would like to avoid.


So...what the the best way to randomly rotate between slaves and ports
in a IO loop?






Thanks

[Josiah Carlson]

If you want a literal round-robin of the servers, don't use
random.choice(), use itertools.cycle().

import itertools
conns = itertools.cycle(r_main_reads)
def execute_get_cookie(guid):
return get_cookie(['my_user_id'], [guid], conns.next())

As for why random.choice() seems to be returning the same object, I
don't know, I'm not seeing that behavior myself.

- Josiah

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>