innobackupex crashes with 'may not be InnoDB datafile or may be corrupted' error
Pavel Shevaev
After power problems in the datacenter(master server powered off all
of a sudden) my master and replicating slave became not synced. I
decided to re-create the slave using innobackupex as follows:
# innobackupex-1.5.1 --stream=tar /tmp/ --slave-info | ssh slave "tar
xfi - -C ~/mysql"
After about a minute this command crashes with the following output:
<skipped>
>> log scanned up to (1008663005500)
>> log scanned up to (1008663286889)
>> log scanned up to (1008663565421)
>> log scanned up to (1008663831324)
innobackupex-1.5.1: Backing up files '/var/lib/mysql/db/*.ibd' (40 files)
>> log scanned up to (1008664127219)
>> log scanned up to (1008664430098)
>> log scanned up to (1008664707455)
>> log scanned up to (1008664991956)
>> log scanned up to (1008665255594)
>> log scanned up to (1008665527730)
>> log scanned up to (1008665769134)
The file 'db/foo.ibd' may not be InnoDB datafile or may be corrupted.
tar_append_tree("db/foo.ibd", "db/foo.ibd"): Input/output error
innobackupex-1.5.1: tar returned with exit code 255.
innobackupex-1.5.1: Error: Failed to stream
'/var/lib/mysql/db/foo.ibd': at /usr/bin/innobackupex-1.5.1 line 336.
This command worked just fine in the very first time when I setup the
slave but now it looks like there is a data corruption problem in the
master, right? However, there are no error entries in the master log
so far...
What would you recommend in this situation?
--
Best regards, Pavel
Alex Yurchenko
recreate your master from it in such a situation? Otherwise what was the
point of slave?
Regards,
Alex
--
Alexey Yurchenko,
Codership Oy, www.codership.com
Skype: alexey.yurchenko, Phone: +358-400-516-011
Pavel Shevaev
<alexey.y...@codership.com> wrote:
> I'm kinda new to this, but aren't you supposed to failover to slave and
> recreate your master from it in such a situation? Otherwise what was the
> point of slave?
You are right and this is what I did in the past but in my situation
master is fine(or pretends to be fine). Switching to a slave,
restoring the master and switching back to it is quite a long
procedure which I'd really to avoid unless there is a good reason.
--
Best regards, Pavel
Jervin R
this bug https://bugs.launchpad.net/percona-xtrabackup/+bug/665210
I'd suggest to try backing up to local disk first then copying to the
slave.
Regards
Jervin
Pavel Shevaev
> Is this table ROW_FORMAT=COMPRESSED?
Nope :(
> I'd suggest to try backing up to local disk first then copying to the
> slave.
Without shutting down the server it doesn't make sense....Copying may
take up to an hour or longer and I can't shut it down for so much
time.
--
Best regards, Pavel
Jervin R
Sorry for the delay.
Do you mean an xtrabackup backup to a local disk is not possible
because it takes long or was it for another reason?
What does 'SHOW TABLE STATUS LIKE 'foo' \G' look like? Also include
SHOW GLOBAL VARIABLES LIKE 'innodb_%';
I noticed you also have an old thread here
http://groups.google.com/group/percona-discussion/browse_thread/threa...
is it the same my.cnf?
On Jul 28, 6:25 pm, Pavel Shevaev <pacha.shev...@gmail.com> wrote:
Pavel Shevaev
> Pavel,
>
> Sorry for the delay.
>
> Do you mean an xtrabackup backup to a local disk is not possible
> because it takes long or was it for another reason?
Looks like the table in question is corrupted :( I tried the
non-stream version of xtrabackup and it also complained about this
particular table. The database server currently works just fine,
reads/writes do happen with this table.
If anyone can recommend any tool which could tell exactly what is
wrong with this table I'll be grateful. I'm scared to run check within
mysql itself since I've experienced a couple of time server crashes
during this procedure when checking corrupted tables.
--
Best regards, Pavel