holland-xtrabackup plugin added

[Andrew Garner]

I just added a simple holland-xtrabackup plugin to master. I just
threw this together over the course of the past hour or so, so there's
bound to be bugs. It is very simple currently, only supports --stream
mode along with the (optional) --slave-info and (optional) --no-lock
options. It's a good start to a more fully featured xtrabackup
plugin.

Feedback welcome.

~Andy

[Tim Soderstrom]

Wow that was fast :)

I'll try this out on my VM perhaps later tonight. Having this would be a great alternative to LVM for some customers who are willing to take the --apply-logs risk. Fantastic work, Andy!

> --
> WIKI: http://wiki.hollandbackup.org
> CODE: http://github.com/holland-backup
> UNSUBSCRIBE: holland-deve...@googlegroups.com
>

[Tim Soderstrom]

Looks good to me except for what appears to be a bug:

#####
[root@holland ~]# holland bk xtrabackup
Holland 0.9.9 started with pid 7417
---> Starting backup run <---
No backups purged
Initiating connection to MySQL
Estimated Backup Size: 229.74MB
Starting backup[xtrabackup/20100531_034007] via plugin xtrabackup
innobackupex-1.5.1 --defaults-file=/var/spool/holland/xtrabackup/20100531_034007/my.xtrabackup.cnf --stream=tar4ibd --slave-info /tmp
Final on-disk backup size 78.70MB 34.26% of estimated size 229.74MB
Backup completed in 46.42 seconds
Uncaught exception while running command 'bk': <exceptions.KeyError instance at 0x2b12050cd7a0>
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/holland/core/command/command.py", line 201, in dispatch
return self.run(self.optparser.prog, opts, *args)
File "/usr/lib/python2.4/site-packages/holland/commands/backup.py", line 96, in run
if not opts.no_lock and config['holland:backup']['lockfile']:
File "/usr/lib/python2.4/site-packages/holland/core/config/configobj.py", line 567, in __getitem__
val = dict.__getitem__(self, key)
KeyError: 'lockfile'
[root@holland ~]# cd /var/spool/holland/xtrabackup/20100531_034007/
[root@holland 20100531_034007]# ls
backup.conf backup.tar.gz my.xtrabackup.cnf xtrabackup.log
[root@holland 20100531_034007]# ls -la
total 80696
drwxrwx--- 2 root root 4096 May 31 03:40 .
drwxrwx--- 3 root root 4096 May 31 03:40 ..
-rw-rw---- 1 root root 402 May 31 03:40 backup.conf
-rw-rw---- 1 root root 82520639 May 31 03:40 backup.tar.gz
-rw-rw---- 1 root root 346 May 31 03:40 my.xtrabackup.cnf
-rw-rw---- 1 root root 5212 May 31 03:40 xtrabackup.log
#####

Holland also does not bail out nicely if Xtrabackup is not installed (should the eventual Xtrabackup RPM just have a dependency for an Xtrabackup RPM?)

Config file looks good though. I can't think of anything I normally use that should be there since we are not doing the apply logs stuff.

[Andrew Garner]

On Tue, Jun 8, 2010 at 10:39 AM, Tim Soderstrom
<t...@moocowproductions.org> wrote:
> Looks good to me except for what appears to be a bug:
>
>
> #####
> [root@holland ~]# holland bk xtrabackup
> Holland 0.9.9 started with pid 7417
> ---> Starting backup run <---
> No backups purged
> Initiating connection to MySQL
> Estimated Backup Size: 229.74MB
> Starting backup[xtrabackup/20100531_034007] via plugin xtrabackup
> innobackupex-1.5.1 --defaults-file=/var/spool/holland/xtrabackup/20100531_034007/my.xtrabackup.cnf --stream=tar4ibd --slave-info /tmp
> Final on-disk backup size 78.70MB 34.26% of estimated size 229.74MB
> Backup completed in 46.42 seconds
> Uncaught exception while running command 'bk': <exceptions.KeyError instance at 0x2b12050cd7a0>
> Traceback (most recent call last):
>  File "/usr/lib/python2.4/site-packages/holland/core/command/command.py", line 201, in dispatch
>    return self.run(self.optparser.prog, opts, *args)
>  File "/usr/lib/python2.4/site-packages/holland/commands/backup.py", line 96, in run
>    if not opts.no_lock and config['holland:backup']['lockfile']:
>  File "/usr/lib/python2.4/site-packages/holland/core/config/configobj.py", line 567, in __getitem__
>    val = dict.__getitem__(self, key)
> KeyError: 'lockfile'

Yeah, I had been reworking the exclusivity/locking behavior of holland
to work better per-backupset. I thought I had fixed this one, but I
guess not. :)

~Andy

[Tim Soderstrom]

I'm also having issues running 'holland mk-config mysqldump-lvm'. I assume that's still being developed?

[Andrew Garner]

On Tue, Jun 8, 2010 at 10:57 AM, Tim Soderstrom
<t...@moocowproductions.org> wrote:
> I'm also having issues running 'holland mk-config mysqldump-lvm'. I assume that's still being developed?

yeah, mysqldump-lvm is a work in progress. This particular case was a
missed commit that I hadn't pushed to master. You should be able to
mk-config for mysqldump-lvm now if you pull the latest, and I think I
fixed the lockfile issues.

~Andy

[Andrew Garner]

I just went through and fixed a few more minor but annoying bugs.
Most of these came from a bad merge from a side branch, caused by a
failed experiment with git-filter-branch and resulting brain-damage on
my part which missed a few critical commits. :) Anyway, I ran
through lvm, mysqldump, mysqldump-lvm and xtrabackup and they should
work reasonably well. xtrabackup and mysqldump-lvm haven't had a lot
of work put into them, though, so YMMV.

~Andy

[Tim Soderstrom]

I thought I would change the subject to keep things separate. mysqldump-lvm looks really cool! I haven't run into any issues with it thus far. A minor issue is the innodb-recovery option for the LVM component? Since mysqldump has to bootstrap a mysqld process anyway, I assume this variable does nothing (it was in the mk-config output which is why I thought I'd bring it out).

Documenting how this plugin works is something we'll definitely want to do since it is a big more complicated than the others but generally makes sense once you figure out what's going on. Need to do more testing of course but it looks pretty solid to me so far.

[Andrew Garner]

Yeah, I just copied and pasted the mysql-lvm plugin and merged in the
mysqldump functionality as a separate plugin. :) I just clipped
innodb-recovery out since it didn't do anything anyway.

~Andy

On Tue, Jun 8, 2010 at 12:11 PM, Tim Soderstrom

[Tim Soderstrom]

Yeah that's what I was wondering about.