Failover script

[JammyZ]

Hi,
   I presume many of you are using redis with a master -> slave configuration and on fail of the master switching to use the slave as the master. From what I've read it seems to be a fairly common setup.
  
  I was wondering if any of you had a script that would detect this situation and automatically trigger the required changes so the downtime is as low as possible. I'll need to write something to do this in the following days but maybe somebody wishes to share his implementation.

Cheers,
Iker.

[Tim Lossen]

iker,

we use a master/slave setup with manual failover.

automatic failover sounds really nice ..... but:
a) this is (hopefully) a very rare event
b) the exact failure conditions (network, power, disk,
memory etc.) are not known in advance
c) the script is extremely hard to test

in the end, the script might actually *compound* the
problem and cause an even longer downtime -- so we figure
we are better off without it.

cheers
tim

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.

--
http://tim.lossen.de

[JammyZ]

Ok, I see your point, but on the other side it means having a way longer downtime.
I wish we had the cluster here already, it would make our lives much easier :)

[Josiah Carlson]

Whether a downtime of a minute vs 10 minutes matters depends on your
business, of course. However, good design is resilient to master
failures (do you have a read-only mode?), and it can be a lot of work
or impossible to pre-code your way to a bullet-proof failover
scenario.

For our 100% uptime stuff, we use haproxy on the frontend to do
load-balancing/failure detection on our web frontends, so if a worker
slave dies, they are automatically pulled from the pool. If the
master dies, certain parts of the site go read-only, someone gets
paged, they figure it out, bring it back up, and re-slave the slaves
one at a time (to ensure that neither the master nor the worker pool
is overloaded at any one time). This has worked for us since July
without issue.

- Josiah