dual master redis setup

[trapni]

Hey all,

I googled for redis in dual-master mode, however, only found hints
that this might not be possible due to the overhead of implementation.

We would like to have a dual master redis to load balance between the
two (and mostly make it more easy to let them be a fallback for each
other!).

Now, that said, I tested it locally, and did test-writes and -reads to
both instances, and it worked.

well, it works, but it is not documented anywhere, and both are marked
as "slave" (becuase I called SLAVEOF on both nodes for each other).

Why is this working? - or - What am I missing?

On the other hand, does anyone have a good practice in _fast_ high-
availablity failover strategy for redis?

Thanks in advance,
Christian Parpart.

[Salvatore Sanfilippo]

Hi,

actually it does not work that well ;) It appears to work, it is just
a side effect of how Redis replication works, but:

1) Expires will not be handled.
2) Things like LPUSH in both sides against the same list with the
wrong timing will create different orders in the two sides.
3) If the link will go down, the two instances will not be albe to reconnect.

We don't support this mode of operation since every dual-master setup
requires some way to perform merge of keys after a netsplit, and since
with async replication it is hard to fix the problem of operations
that change a value.
We are looking for ways to support an easy HA setup without requiring
to run Redis Cluster, but there is currently not a solution as there
are other things with higher priority (2.6 release and finishing 3.0
with cluster support).

For now you can use what other users are using, that is HAProxy, or a
script that checks INFO and performs the election of the slave when
needed (you need a way to reconfigure your clients to point to the
slave or to swap IP addresses). No silver bullet...

Cheers,
Salvatore

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware

http://invece.org
"We are what we repeatedly do. Excellence, therefore, is not an act,
but a habit." -- Aristotele

[Christian Parpart]

We're using HA in front, yeah, however, setting up one node as backup node (which is the slave of the master/active node) would lead (IMHO) into a problem, that is, when the original master comes back up, having an empty or partial database, which is something I'd like to prevent.

My proposal (looking for comments) is as follows:

setting up keepalived on two nodes, sharing the same virtual IP, using a custom check script on both nodes, to test whether or not redis-server is running properly.

if now the master host is down (either redis alone or the host completely), the other host takes over (via keepalived) and adds the shared virtual IP to its local interface.

if the slave takes over the shared IP, the keepalived script also issues a "SLAVEOF NO ONE" via netcat to its local redis, thus, making it the master effectively.

Now, if the original master host comes up again, keepalived decides to switch back the shared IP, and then I'd like to let the temporary master (original slave) sync all changes back to the primary master.

however, especially the latter part becomes quite complicated to implement (administration-wise) because you cannot switch the VRRP-IP immediately since it does not yet have all data diff synced back.

Maybe there is some way to first let redis sync and *then* switch the IP.

Any objections?

Christian Parpart.