wsrep_sst_xtrabackup-v2 failing on MariaDB 10.2

[Tim]

Hi folks

I'd appreciate some pointers on what I might have done wrong here.

I'm running a Galera Cluster under MariaDB 10.2.6

Adding a new node with rsync is working fine, but I won't be using that in production, so I am trying with xtrabackup-v2 (2.4.7)

Below is the output in /var/log/messages when I try to add a fresh node. There's a lot of output before this, but this snippet shows the first error.

I can post more info, but there's a lot of it and it looks fairly normal up to this point. What could be causing the "xtrabackup_checkpoints missing" error?

Jun  2 17:49:26 localhost mysqld: WSREP_SST: [INFO] Waiting for SST streaming to complete! (20170602 18:49:26.806)

Jun  2 17:49:27 localhost mysqld: 2017-06-02 18:49:27 139883502876416 [Note] WSREP: (d0cc2a25, 'tcp://0.0.0.0:4567') connection to peer d0cc2a25 with addr tcp://10.0.0.4:4567 timed out, no messages seen in PT3S

Jun  2 17:49:27 localhost mysqld: 2017-06-02 18:49:27 139883502876416 [Note] WSREP: (d0cc2a25, 'tcp://0.0.0.0:4567') turning message relay requesting off

Jun  2 17:49:36 localhost mysqld: WSREP_SST: [ERROR] xtrabackup_checkpoints missing, failed innobackupex/SST on donor (20170602 18:49:36.717)

Jun  2 17:49:36 localhost mysqld: WSREP_SST: [ERROR] Cleanup after exit with status:2 (20170602 18:49:36.721)

Jun  2 17:49:36 localhost mysqld: 2017-06-02 18:49:36 139883486091008 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.0.0.4' --datadir '/mnt/mysql/data/'   --parent '8900'  '' : 2 (No such file or directory)

Jun  2 17:49:36 localhost mysqld: 2017-06-02 18:49:36 139883486091008 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script.

Jun  2 17:49:36 localhost mysqld: 2017-06-02 18:49:36 139883832133760 [ERROR] WSREP: SST failed: 2 (No such file or directory)

Jun  2 17:49:36 localhost mysqld: 2017-06-02 18:49:36 139883832133760 [ERROR] Aborting

Jun  2 17:49:36 localhost mysqld: 2017-06-02 18:49:36 139883494483712 [Warning] WSREP: 0.0 (dev1): State transfer to 1.0 (dev2) failed: -22 (Invalid argument)

Jun  2 17:49:36 localhost mysqld: 2017-06-02 18:49:36 139883494483712 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():736: Will never receive state. Need to abort.

[Tim]

Solved it minutes after posting, despite having me stumped for hours.

The "sstuser" account was missing the "PROCESS" privilege. Absent from some of the guides I've read, but documented here.