need help with MariaDB+xtrabackup+compress

[Neal K]

Hi,

I am using MariaDB-Galera-server-5.5.29 and xtrabackup-2.1.3 , I have SST with xtrabackup working fine. But I cannot get it to work if I enable compression. I've tried adding settings to my.cnf, patching the script files etc..

I'd like to get compression working since I am syncing across data centers, when I added '--compress-level=8' to the rsync scripts it sped things up 3x

Thanks.

[Raghavendra D Prabhu]

Hi Neal,

The compression involving xtrabackup/innobackupex doesn't work
OOTB when it comes to streaming. Few modifications are required.

The wsrep_sst_xtrabackup packaged in PXC 5.5.31 should support
compression. You can look at its documentation in the script
itself - (You should be able to use it with MariaDB Galera).
Specifically to support compression you need to add compress
under [xtrabackup] and use xbstream.

However, it has been improved further in 5.5.33 PXC (unreleased). You can take a look at wsrep_sst_xtrabackup here - https://bazaar.launchpad.net/~percona-core/percona-xtradb-cluster/5.5/files/head:/Percona-Server/scripts/


You can also check the documentation here - http://www.percona.com/doc/percona-xtradb-cluster/manual/xtrabackup_sst.html

If you need example configuration you can check: https://bazaar.launchpad.net/~percona-core/percona-xtradb-cluster/5.5/files/head:/tests/conf/

>
>Thanks.
>
>--
>You received this message because you are subscribed to the Google Groups "codership" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to codership-tea...@googlegroups.com.
>For more options, visit https://groups.google.com/groups/opt_out.




Regards,
--
Raghavendra Prabhu | http://about.me/raghavendra.prabhu
Product Lead | Percona XtraDB Cluster (PXC)
Percona LLC. - http://www.percona.com / Blog: http://www.mysqlperformanceblog.com/
Contact: http://wnohang.net/contact | GPG: 0xD72BE977

Make plans to attend Percona Live London MySQL Conference
2013.<http://www.percona.com/live/london-2013/>

[Neal K]

Hi Raghavendra,

 Thanks for the reply but I'm still unable to get it working

First I tried using 5.5.33, I grabbed wsrep_sst_common.sh and   wsrep_sst_xtrabackup.sh. Then I installed PV and SOCAT packages.

However it didn't work , I kept getting

130815 15:40:47 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'donor' 

--address '10.97.130.106:444/xtrabackup_sst' --auth 'xx:xx' --socket '/var/lib/mysql/mysql.sock' 

--datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --gtid '50c9d570-ff37-11e2-0800-3aecc77d219c:13426': 22 (Invalid argument)

I'm guess MariaDB didn't call xtrabackup_sst with the correct arguments

Then I tried 5.5.31, replacing wsrep_sst_common.sh and   wsrep_sst_xtrabackup.sh.

That worked when I had no special settings in my.cnf.

When I added the following to my.cnf :

[xtrabackup]

parallel=8

[sst]

streamfmt=xbstream

transferfmt=socat

sockopt=,nodelay,sndbuf=1048576

I got this error

130816 11:30:13 [Note] WSREP: Requesting state transfer: success, donor: 0

^Gxbstream: Can't create/write to file './ibdata1' (Errcode: 17)

xbstream: failed to create file.

WSREP_SST: [ERROR] Xbstream failed (20130816 11:30:14.149)

WSREP_SST: [ERROR] Data directory /var/lib/mysql/ needs to be empty for SST: lp:1193240 Manual intervention required in that case (20130816 11:30:14.152)

WSREP_SST: [ERROR] Cleanup after exit with status:32 (20130816 11:30:14.154)

WSREP_SST: [ERROR] Killing nc pid  (20130816 11:30:14.161)

WSREP_SST: [INFO] Removing the sst_in_progress file (20130816 11:30:14.162)

130816 11:30:14 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'joiner' --address '10.97.130.106' --auth 'xx:xx' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --parent '3341': 32 (Broken pipe)

I tried rm -rf /var/lib/mysql/* , and then starting mysql but got the same error.

[Matthew Casperson]

Did you ever manage to get compression working?