Galera rejoin failure node

[Nguyen Truong Son]

Hi all

I have 3 mysql galera cluster nodes: Node 1, Node 2, Node 3.

Mysql version 8.0.21. Centos 7.

All my 3 nodes are down. 

Then I want to start the cluster. I determine that node 1 is the node that has newest data.

But by mistake, I start node 2 first with /usr/bin/mysqld_bootstrap then stop it.

And then I start node 1 with /usr/bin/mysqld_bootstrap and join node 3 successfully.

But when I start node 2 with systemctl start mysqld, it always start with a different cluster.

Now I have 2 cluster:

1. Node 1, node 3

2. Node 2

How can I do to force node 2 sync and join the cluster again?

Thank you very much.

[mohsen shahbazi]

you can remove whole data directory of mysql ( based on defaults it would be /var/lib/mysql/ ) and try to join your node again to cluster . it will try to make an SST snapshot transfer from one of other nodes ( donor node ) and got all data again .
just keep in mind that donor node will not accept any write query in duration of SST snapshot transfer and will keep write/update queries in queue until this operation is finished .

[Chachia Mohamed]

1. connect to one of the working nodes, and check " mysql -u root -p  "
2. type  SHOW GLOBAL STATUS LIKE 'wsrep_%'; 
   
3. copy the wsrep_cluster_state_uuid value and copy the wsrep_last_committed value 
4. now connect to the node you want it to join and paste those values in /var/lib/mysql/grastate.dat in uuid , and seqno 
5. try now to restart mariadb service systemctl restart mariadb
6. now on the previous node we go to mysql prompt again and type SET GLOBAL wsrep_cluster_address="gcomm://ip of node1,ip of node 2 , ip of node 3";
   
7. now we go back to the joiner node and restart mariadb service systemctl restart mariadb