innobackupex output to logfile

[Kari Lehtinen]

Hi,

I'm planning to move from mysqldump to innobackupex using stream method and gzip, however I'm not able to forward output to logfile. I have tried e.g. following but file /tmp/xtrabackup.log stays empty.

innobackupex --user=root --stream=tar /backups/ | gzip --fast -c - > /backups/backup.tar.gz > /tmp/xtrabackup.log

Any suggestions how to redirect innobackupex output to logfile ?

Installed version: percona-xtrabackup               2.1.6-702-1.precise


Thanks,
Kari

[Jaime Crespo]

Please note that with the stream option you are outputting your backups to the standard output, so it cannot be used for logging. Use the error output (#2) instead:

innobackupex --user=root --stream=tar /backups 2> /tmp/xtrabackup.log | gzip --fast -c - > /backups/backup.tar.gz

Make sure that /backups is on a remote filesystem or you copy it elsewhere for a correct backup process.

Please also note that streaming backups have already compression support, and it has the advantage over 3rd party methods that allow on-the-fly compression, parallel backups and multiple compression threads (it had to be implemented in xbstream due to tar limitations):

innobackupex --compress --parallel=4 --compress-threads=4 --stream=xbstream /backups > /backups/backup.xbstream 2> /tmp/xtrabackup.log

To later decompress:

xbstream -x -C /backups < /backups/backup.xbstream
innobackupex --decompress /backups

For more details about the advantages of in-house compression and parallel backups:

<http://www.percona.com/live/mysql-conference-2013/sessions/fine-tuning-percona-xtrabackup-your-workload>

Regards,

--
You received this message because you are subscribed to the Google Groups "Percona Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to percona-discuss...@googlegroups.com.
To post to this group, send email to percona-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Jaime Crespo
Senior MySQL Instructor at Percona
Public training schedule: http://www.percona.com/training/

Phone: +1 888 401 3401 ext. 553
Skype: jynusx

[ad...@extremeshok.com]

for completeness how would one restore the backup ? Decompress then restore or can it be done in a single command ?

Sent from my iPhone 5

[Kari Lehtinen]

Thanks, this seems to work. I'm using gzip because it seems to provide slightly better compression ratio over qpress and speed is not the issue in this case.

Regards,
Kari

On Monday, December 30, 2013 11:38:33 AM UTC+2, Jaime Crespo wrote:

Please note that with the stream option you are outputting your backups to the standard output, so it cannot be used for logging. Use the error output (#2) instead:

innobackupex --user=root --stream=tar /backups 2> /tmp/xtrabackup.log | gzip --fast -c - > /backups/backup.tar.gz

Make sure that /backups is on a remote filesystem or you copy it elsewhere for a correct backup process.

Please also note that streaming backups have already compression support, and it has the advantage over 3rd party methods that allow on-the-fly compression, parallel backups and multiple compression threads (it had to be implemented in xbstream due to tar limitations):

innobackupex --compress --parallel=4 --compress-threads=4 --stream=xbstream /backups > /backups/backup.xbstream 2> /tmp/xtrabackup.log

To later decompress:

xbstream -x -C /backups < /backups/backup.xbstream
innobackupex --decompress /backups

For more details about the advantages of in-house compression and parallel backups:

<http://www.percona.com/live/mysql-conference-2013/sessions/fine-tuning-percona-xtrabackup-your-workload>

Regards,

On Mon, Dec 30, 2013 at 8:57 AM, Kari Lehtinen <kar...@gmail.com> wrote:

Hi,

I'm planning to move from mysqldump to innobackupex using stream method and gzip, however I'm not able to forward output to logfile. I have tried e.g. following but file /tmp/xtrabackup.log stays empty.

innobackupex --user=root --stream=tar /backups/ | gzip --fast -c - > /backups/backup.tar.gz > /tmp/xtrabackup.log

Any suggestions how to redirect innobackupex output to logfile ?

Installed version: percona-xtrabackup               2.1.6-702-1.precise


Thanks,
Kari

--
You received this message because you are subscribed to the Google Groups "Percona Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to percona-discussion+unsub...@googlegroups.com.

To post to this group, send email to percona-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jaime Crespo]

As I wrote the decompression method the end of my last email. This is the full process:

Unpack the single transmitted file:

xbstream -x -C /backups < /backups/backup.xbstream

Decompress:

innobackupex --decompress /backups # you can use --parallel for faster decompression

Prepare:

innobackupex --apply-log /backups

Stop mysql (assuming a full restore):

service mysql start

 

Restore:

innobackupex --copy-back /backups # change --copy-back to --move-back if you can do it faster (same partition) and have a copy of the backup

Change permissions:

chown -R mysql:mysql $YOURDATADIR

Start mysql:

service mysql start

-------------

I have done these on different commands for ease, you can do several of these in parallel (compression uses standard quicklz from qpress) but decompression and preparation (apply-log) must be done on separate steps.

[Jaime Crespo]

On Mon, Dec 30, 2013 at 11:01 AM, Kari Lehtinen <kar...@gmail.com> wrote:

Thanks, this seems to work. I'm using gzip because it seems to provide slightly better compression ratio over qpress and speed is not the issue in this case.

Yes, that is a good option, as the compression option is optimised for fast compression with low resource usage and more flexibility (individual compress files) at the cost of lower compression ratio. Ej:

Original file: 4.4GB
Compressed with --compress (quicklz, one per file): 2.5G
Compressed with tar-gzip: 1.5G
Compressed with tar-bz2: 1.1G
Compressed with 7za (7z format): 688M

The speed difference, however, is huge, and only the first one can be considered "real time backups".