Extending the vertx-redis-client with a failover feature

[Florian Kammermann]

I would like to extend the vertx-redis-client with a failover feature.

Sadly we can't use redis cluster or sentinel, so I have to implement it my own.

Now I try to find a way to extend the vertx-redis-client in a non invasive way.

What I need, is the information, that the client got disconnected.

At this point I would like to run my own watchdog and make a failover (reconfigure the client) to a slave, if needed.

The best I came up with is to publish the state of RedisConnection on the EventBus.

To reach that, I have to touch RedisOptions, RedisConnection, AbstractRedisClient, RedisClientImpl and RedisClient.

Any suggestions how I could reach the goal, of getting the information if the client is connected or not, in way where I have to touch less code?

Thanks for your suggestions

Florian

[Julien Viet]

Hi,

do you want to failover on connection failure (i.e try another server rather than the current one) or when the connection has been closed ?

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/3515e16c-2c58-444c-9876-905afe626c8b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Florian Kammermann]

I did a quick test:

1. connect client 
2. set a breakpoint 
3. shutdown redis
4. let the code execute

In the log I've seen this: Connection closed!

I the code there are 2 sections where this is logged, both time the state is set to State.DISCONNECTED

So I thought I have to react on State.DISCONNECTED

I think about something like this:

• provide master/slave host/port over the config
  
• when the master falls into the state State.DISCONNECTED, try to establish a connection for eg. 30sec
  
• if not successfull, propagate the slave to the master 
  
• reconfigure the redis client
  

Additional read requests could be distributed for better performance.

[Julien Viet]

I’m not sure whether you are suggesting to add new features to the Redis client or adding new callbacks (like a disconnected callback) to build on top of Redis client this load balancing policy.

can you clarify ?

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/014daf15-ce6a-44e4-b60f-ec561dbefae8%40googlegroups.com.

[Florian Kammermann]

1. I wan't to build this new feature (*failover)
   
2. I want to build it in a way, that I have not to migrate a lot, when upgrade to a newer version of vertx-redis-client
3. Where this feature will be built has to be defined
1.  outside in its own project with vertx-redis-client as dependency
2. fork of vertx-redis-client and built in feature

* Definition of failover:

• starting point: 
• we have a redis master and slave 
• the redis client(s) is connecting to the master
• the redis client knows the host/port of the slave
• if the master becomes unavailable (what unavailable is hast to be defined), the client propagates the slave to the master and connects to the new master

[Julien Viet]

I think we could at least provide in the RedisClient a notification when the client is disconnected (either directly or when connected).

Is the list of redis server dynamic then in your case ? or you just have two servers (master and slave). Now there is a single server specified in the options, there could be more servers but that would remain static.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/583ca226-4ea3-45e5-9a7d-0a4189dd3ce5%40googlegroups.com.

[Florian Kammermann]

> Is the list of redis server dynamic then in your case ? or you just have two servers (master and slave).

Just two servers (master and slave)

We now investigating after all in redis sentinel.

So if redis sentinel works for us, I will try to extend the vertx-redis-client with the ability of redis sentinel.

So for the moment its OK, will come back, if we still need this possibility of notification.

Thanks for your help.

[Paulo Lopes]

Redis client reconnects on connection error to the same host. There is a pr for sentinel support, however is has a huge merge conflict that needs to be picked up either by one of the developers or the community.