Restart first node Galera 4 and Mysql 8 on Centos 7

[Nguyen Truong Son]

Dear all

I use Galera 4 and Mysql 8 on Centos 7.

I have 3 nodes: gcomm://10.38.29.9,10.38.29.99,10.38.29.212

My first node to startup a bootstrap is 10.38.29.9.

One day, node 10.38.29.9 get error:

2021-07-03T11:01:00.903041Z 1 [Warning] [MY-010584] [Repl] Slave SQL: Error executing row event: 'Deadlock found when trying to get lock; try restarting transaction', Error_code: MY-001213

2021-07-03T11:01:00.915301Z 1 [Warning] [MY-000000] [WSREP] Event 7 Write_rows apply failed: 1, seqno 37050322

2021-07-03T11:01:00.994384Z 1 [ERROR] [MY-000000] [Server] WSREP: Inconsistency detected: Inconsistent by consensus on be0acc17-27e3-11eb-86ad-5befdf207edd:37050322

at galera/src/replicator_smm.cpp:process_apply_error():1347

2021-07-03T11:01:01.026450Z 1 [ERROR] [MY-000000] [Server] WSREP: Failed to apply write set: gtid: be0acc17-27e3-11eb-86ad-5befdf207edd:37050322 server_id: 5a662628-4410-11eb-b784-a336924c6c06 client_id: 3094139 trx_id: 666910139 flags: 3 (start_transaction | commit)

And when I try to restart node 10.38.29.9 with : 

systemctl restart mysqld

It can not start with error:

2021-07-04T16:18:54.795888Z 0 [ERROR] [MY-000000] [Server] WSREP: It may not be safe to bootstrap the cluster from this node. It was not the last one to leave the cluster and may not contain all the updates. To force cluster bootstrap with this node, edit the grastate.dat file manually and set safe_to_bootstrap to 1 .

2021-07-04T16:18:54.795994Z 0 [ERROR] [MY-000000] [WSREP] wsrep::connect(gcomm://10.38.29.9,10.38.29.99,10.38.29.212) failed: 7

2021-07-04T16:18:54.819629Z 0 [ERROR] [MY-010119] [Server] Aborting

What can I do to start and rejoin the node? I've searched but its not successful.

Thanks.

[Nguyen Truong Son]

Can I try this solution? https://microdevsys.com/wp/error-wsrep-it-may-not-be-safe-to-bootstrap-the-cluster-from-this-node-it-was-not-the-last-oneave-the-cluster-and-may-not-contain-all-the-updates/

Does it make the data in inconsistant state?

I tried to set another node  10.38.29.99 as  Bootstrap: SET GLOBAL wsrep_provider_options='pc.bootstrap=YES';

https://galeracluster.com/library/documentation/quorum-reset.html

And removed all mysql-wsrep* then reinstalled them but I was unsuccessful.

I dont know what to do to make it run again.

Please help me. 

Thanks.

[alexey.y...@galeracluster.com]

Hi,

The problem is that you are trying to bootstrap a NEW cluster with the
node that just failed due to inconsistency. If you have the remaining 2
nodes running and want to join then with a 3rd node - you just need to
start it as a regular node. Just like you started nodes 2 and 3 before
that

Bootstrap option is specifically to bootstrap a new cluster. That's the
only time it should be used.

Kind regards,
Alex

[Nguyen Truong Son]

Thank you so much Mr Alexey.

The command line I used to start node  10.38.29.9 is 

systemctl restart mysqld

It's the command line to start regular node.

But it still runs as bootstrap node.

Can you help me with this?

[Nguyễn Trường Sơn]

Hi All.

Can anyone help me with this? The answer of Mr Alexey did not help me to resolve this issue.

Virus-free. www.avast.com

--
You received this message because you are subscribed to a topic in the Google Groups "codership" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codership-team/Sem7cG5kgC4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codership-tea...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codership-team/dd49162f-20dd-4198-9139-bbd3245a0909n%40googlegroups.com.

--

**************************************
Nguyễn Trường Sơn
Mobile: 0904010635

[Nguyễn Trường Sơn]

Hi Brian

Thank you for your answer.

I have two surviving nodes in the cluster:  (10.38.29.99,10.38.29.212)

State of 2 nodes is primary now.

My bootstrap node of Original Cluster is 10.38.29.9. And now it is at bad state.

I can restart 10.38.29.99 or 10.38.29.212, but there are only 2 nodes in cluster now.

I need to rejoin the 10.38.29.9.

On Wed, Aug 25, 2021 at 3:13 PM brian ryberg <brian....@googlemail.com> wrote:

Do you still have the two surviving nodes running the original cluster ?   (10.38.29.99,10.38.29.212) 
(Edit: No, I believe you now only have    10.38.29.212, since it seems you tried to force the  10.38.29.99 to bootstrap.)

Does the last node  10.38.29.212  run the originally started cluster? 

Can the two bad nodes "see" this node? 

If so, you *should* be able to do systemctl start mysql on those bad nodes, and they should do a full refresh from the running .212 node.... 

In a perfect world. 

You received this message because you are subscribed to the Google Groups "codership" group.
To unsubscribe from this group and stop receiving emails from it, send an email to codership-tea...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codership-team/CA%2B72v%3DrPqNH96FjKpksbHNk3Ef23yqAGgyq4dP6G9uK-4H7aWA%40mail.gmail.com.