Traditional replication and mysql-galera/wsrep

[Kian]

Hello!

I currently have a 3-node Galera cluster. The first node is receiving
replication data from a separate (non-galera) MySQL server, using
traditional MySQL replication. No other writes are happening because
this is a test cluster.

On node1, I have wsrep_cluster_address="gcomm://". On node2 and
node3, I have the cluster address set to the address of node1.

Over the weekend, it appears MySQL on node1 crashed (out of memory)
but mysqld_safe restarted it. When I checked this morning, I noticed
a wsrep_cluster_size of '1' on node1, and wsrep_cluster_size of '2' on
node2 and node3. In other words, node1 appeared to be disconnected
from the cluster.

However, replication still seemed to be working (IO/SQL threads
running, 0 seconds behind master).

My questions:

1) This sounds like a split-brain problem, because I don't believe any
of the writes (via replication) could have been propagated to node2 or
node3, but they still succeeded on node1. How could this happen, with
3 nodes in the cluster?

2) When this happens, what should I do? Should I set
wsrep_cluster_address on node1 to the address of either node2/node3?
In this case, I will be losing data, correct?

3) After initially setting up a cluster, should I change the
wsrep_cluster_address of the primary node, instead of leaving it as an
"empty" gcomm://?

Thanks for all of your help.

[Alex Yurchenko]

Hi Kian,

This is not split brain problem, this is a new cluster problem.
gcomm:// on startup means "create a new cluster". Old nodes are unaware
of it, for them that node just died.

> 2) When this happens, what should I do? Should I set
> wsrep_cluster_address on node1 to the address of either node2/node3?
> In this case, I will be losing data, correct?

In this particular case you should reconnect nodes 2/3 to new cluster
formed by node1.

> 3) After initially setting up a cluster, should I change the
> wsrep_cluster_address of the primary node, instead of leaving it as
> an
> "empty" gcomm://?

You should do exactly that.

> Thanks for all of your help.

Regards,
Alex

--
Alexey Yurchenko,
Codership Oy, www.codership.com
Skype: alexey.yurchenko, Phone: +358-400-516-011