multimaster using redis

[Dimi Mate]

Good Day

Couple of days ago we decided to add a second salt-master to our cluster mainly for migration reasons. The main idea was to be able to manage all minions from both masters. After going through the docs and applying the relevant configurations i still dont understand a few functionalities and concepts.

I used redis as the caching backend and the final setup is as follows:

- salt-master1 running redis master.

- salt-master2 running redis slave server which replicates from salt-master1 primary redis instance.

- server60 (minion) to operate with two masters.

All works fine however i noticed the following:

1) when running a state that generates a config file with a list of servers gathered from saltmine, it only adds "server60", that is, the only minion in the cluster that set to operate with multimaster.

Is this behavior normal? I understand that the relevant minion is configured to operate with multiple masters however, all other servers still send their data (grains e.t.c) to salt-master1 despite the fact that are not configured with two masters. Is that correct? If yes, why only the multimaster minion "can be gathered" from saltmine?

2) When i execute a state on the second saltmaster (salt-master2), i see in log file a message (piped from redis) that you cannot write to a slave redis instance. 

What i understand from the docs, in order to run with multiple masters, cached data must be shared on both masters. However, how will the second master work if it cannot write to its cache. Have i understood something wrong?

Thanks

[Kevin Landreth]

Just run redis on a single instance or on an external host from the salt masters.  It is only a cache so if you lose it, it's not the end of the world... but you need exactly one instance to connect to.

Redis HA is a real pain (trib, sentinel, redis-cluster, floating VIP, .....) when running on only two hosts.  It's doable and up to you if it's worth it.  But if you can spare a VM or host somewhere to host redis, I'd go that route.