Galera cluster SSL failure

[Doug Bell]

I am attempting to establish a new three-node Galera cluster on top of MariaDB 10.2.12 using SSL for all communications.  I have a local private certificate authority that has generated certificates for each of the nodes in the cluster.

I am able to start the first node of the cluster (10.0.7.1) with galera_new_cluster successfully and apparently listening on SSL ports.

Mar  8 13:22:53 mariadb-node1 mysqld: 2018-03-08 13:22:53 140230674102400 [Note] WSREP: (5ef0dfb7, 'ssl://0.0.0.0:4567') listening at ssl://0.0.0.0:4567
Mar  8 13:22:53 mariadb-node1 mysqld: 2018-03-08 13:22:53 140230674102400 [Note] WSREP: (5ef0dfb7, 'ssl://0.0.0.0:4567') multicast: , ttl: 1


However when I start the second node (10.0.7.2), the process fails to start and I get the following errors on the first node:

Mar  8 13:24:01 mariadb-node1 mysqld: 2018-03-08 13:24:01 140229944010496 [ERROR] WSREP: handshake with remote endpoint ssl://10.0.7.2:48364 failed: asio.ssl:336105606: 'certificate verify failed' ( 336105606: 'error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed')
Mar  8 13:24:02 mariadb-node1 mysqld: 2018-03-08 13:24:02 140229944010496 [ERROR] WSREP: handshake with remote endpoint ssl://10.0.7.2:48370 failed: asio.ssl:336105606: 'certificate verify failed' ( 336105606: 'error:14089086:SSL routines:ssl3_get_client_certificate:certificate verify failed')


Here is the primary configuration file /etc/my.cnf.d/server.conf.  Each of the other nodes is identical except for the local certificate and node name.

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
ssl-ca=/etc/pki/mysql/ca-chain.cert.pem
ssl-cert=/etc/pki/mysql/mariadb-node1.cert.pem
ssl-key=/etc/pki/mysql/mariadb-node1.key.pem

[galera]
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_provider_options = "gcache.size=512M; gmcast.segment=0; socket.ssl_cert=/etc/pki/mysql/mariadb-node1.cert.pem; socket.ssl_key=/etc/pki/mysql/mariadb-node1.key.pem; socket.ssl_ca=/etc/pki/mysql/ca-chain.cert.pem;"
wsrep_cluster_name="core1"
wsrep_cluster_address="gcomm://10.0.7.1,10.0.7.2,10.0.7.3"
wsrep_sst_method=rsync
wsrep_node_name="mariadb-node1.mynetwork.local"
wsrep_node_address="10.0.7.1"
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0
wsrep_slave_threads=1
innodb_flush_log_at_trx_commit=0

I am focusing on getting the cluster up and running and not on encrypting the client communications (yet).  The certificates validate against the CA certs (root/intermediate) correctly, and are set with the DN matching each server name.

What could be causing this issue?

[Aura Muccio]

Hi - did you ever get this resolved? I am facing the exact same issue - same error messages, etc. 

[Matthew East]

I also ran into this issue. I was able to resolve it by recreating my certificates with both TLS Web Server Authentication and TLS Web Client Authentication in the Extended Key Usage extension.