can't bootstrap the first node in my galera cluster

[Christopher Marshall]

I am setting up galera clustering for the first time on the primary node and can't get past the first run with the wsrep_new_cluster config parametr present.

After the upgrade from mysql to galera, and adding the wsrep_new_cluster parameter, I started mysql through systemctl
systemctl start mysql

And it failed to start.  Running systemctl status mysql generates this report:
● mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─filelimit.conf, migrated-from-my.cnf-settings.conf
   Active: failed (Result: exit-code) since Fri 2016-07-29 17:04:35 UTC; 50s ago
  Process: 3730 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 3702 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 6743 ExecStartPre=/bin/sh -c VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ] &&   systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=1/FAILURE)
  Process: 6741 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 Main PID: 3702 (code=exited, status=0/SUCCESS)

Jul 29 17:04:31 em7cu1 systemd[1]: Starting MariaDB database server...
Jul 29 17:04:35 em7cu1 sh[6743]: WSREP: Failed to recover position: '2016-07-29 17:04:32 139857692207232 [Note] /usr/sbin/mysqld (mysqld 10.1.16-MariaDB) starting as process 6908 ...'
Jul 29 17:04:35 em7cu1 systemd[1]: mariadb.service: control process exited, code=exited status=1
Jul 29 17:04:35 em7cu1 systemd[1]: Failed to start MariaDB database server.
Jul 29 17:04:35 em7cu1 systemd[1]: Unit mariadb.service entered failed state.
Jul 29 17:04:35 em7cu1 systemd[1]: mariadb.service failed.

So it seems that /usr/bin/galera_recovery is complaining with this message:
WSREP: Failed to recover position: '2016-07-29 17:04:32 139857692207232 [Note] /usr/sbin/mysqld (mysqld 10.1.16-MariaDB) starting as process 6908 ...'

Here are the relevant config parameters I set in my /etc/my.cnf:
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
log_bin=binlog
binlog_format=ROW
bind-address=0.0.0.0
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://192.168.38.207,192.168.38.203"
wsrep_cluster_name='cg1_cluster'
wsrep_node_address='192.168.38.207'
wsrep_node_name='cu1'
wsrep_sst_method=rsync
wsrep_sst_auth=cluster:password

I upgraded from mysql to galera using this procedure:
                systemctl stop mysql
                (edit /etc/my.cnf so that: wsrep_on=OFF)
                systemctl start mysql
                mysql_upgrade
                systemctl stop mysql
                (edit /etc/my.cnf, so that: wsrep_on=ON, wsrep-new-cluster)
                systemctl start mysql

I was having trouble passing command line parameters like --wsrep-new-cluster through systemctl so I would just edit /etc/my.cnf between commands

Any insight into what I am going wrong would be appreciated.

Chris Marshall

[Philip Stoev]

Hello,

To bootstrap a new cluster, please try the dedicated script provided by
MariaDB for the purpose: galera_new_cluster.sh

Thank you.

Philip Stoev

--
You received this message because you are subscribed to the Google Groups
"codership" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to codership-tea...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Christopher Marshall]

I tried running galera_new_cluster.sh and had the same result.

I believe the issue is this bug:
https://jira.mariadb.org/browse/MDEV-10420

In the bug report, the author was able to get around this by not starting through sytsemctl, but by running mysqld directly in 10.1.16.  They also got it to work by downgrading to 10.1.14.

I am running mariadb 10.1.16.  A colleague of mine at work, who ran into the same problem was able to get a galera cluster up by running safe_mysqld directly instead of going through systemctl,