Re: dump | restore fails: unknown tape header type 1853384566
Mikhail T.
> On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote:
>
>> I'm trying to migrate a filesystem from one disk to another using:
>>
>> dump a0hCf 0 32 - /old | restore -rf -
>>
>> (/old is already mounted read-only). The process runs for a while and
>> then stops with:
>>
>> [...]
>> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009
>> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009
>> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009
>> unknown tape header type 1853384566
>> abort? [yn]
>>
>> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's
>> dump's output?
>>
>
> What happens if you don't use the cache?
>
No big difference:
dump a0f - /old | restore -rf -
[...]
DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009
DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009
DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009
unknown tape header type -621260722
abort? [yn]
Looks like a junk value somewhere... Unitialized variable or some such.
-mi
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-...@freebsd.org"
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
Mikhail T.
>> Daniel O'Connor написав(ла):
>>
>> On Tuesday 24 March 2009 11:55:07 Mikhail T. wrote:
>>
>>
>>
>>>> I'm trying to migrate a filesystem from one disk to another using:
>>>>
>>>> dump a0hCf 0 32 - /old | restore -rf -
>>>>
>>>> (/old is already mounted read-only). The process runs for a while and> >> then stops with:
>>>>
>>>> [...]
>>>> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009
>>>> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009
>>>> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009
>>>> unknown tape header type 1853384566> >> abort? [yn]
>>>>
>>>> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's
>>>> dump's output?
>>>>
>>>>
>>>>
> > What happens if you don't use the cache?
>
>>>
>>>
>> No big difference:
>>
>>> dump a0f - /old | restore -rf -
>>>
>> [...]
>> DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009
>> DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009
>> DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009
>> unknown tape header type -621260722> abort? [yn]
>>
>> Looks like a junk value somewhere... Unitialized variable or some such.
>>
>>
> dump a0f some.file /old (or dump 0f - /old | gzip -c > file.dump.gz)
> restore rf some.file
>
> danny
>
Well, the first part (the dump) runs almost to the completion, but hangs
at the very end for some reason:
dump 0aCf 64 /ibm/ibmo.0.2009-03-24.dump /old
DUMP: WARNING: should use -L when dumping live read-write filesystems!
DUMP: Date of this level 0 dump: Tue Mar 24 05:59:27 2009
DUMP: Date of last level 0 dump: the epoch
DUMP: Dumping /dev/ad2s1e (/ibmo) to /ibm/ibmo.0.2009-03-24.dump
DUMP: mapping (Pass I) [regular files]
DUMP: Cache 64 MB, blocksize = 65536
DUMP: mapping (Pass II) [directories]
DUMP: estimated 152357442 tape blocks.
DUMP: dumping (Pass III) [directories]
DUMP: dumping (Pass IV) [regular files]
DUMP: 0.83% done, finished in 9:59 at Tue Mar 24 16:04:19 2009
DUMP: 2.74% done, finished in 5:55 at Tue Mar 24 12:05:07 2009
DUMP: 4.66% done, finished in 5:06 at Tue Mar 24 11:21:27 2009
DUMP: 6.58% done, finished in 4:43 at Tue Mar 24 11:03:37 2009
...
DUMP: 91.54% done, finished in 0:23 at Tue Mar 24 10:38:15 2009
DUMP: 93.41% done, finished in 0:18 at Tue Mar 24 10:38:02 2009
DUMP: 95.27% done, finished in 0:13 at Tue Mar 24 10:37:50 2009
DUMP: 97.15% done, finished in 0:07 at Tue Mar 24 10:37:36 2009
DUMP: 99.03% done, finished in 0:02 at Tue Mar 24 10:37:23 2009
DUMP: DUMP: 152769349 tape blocks on 1 volume
DUMP: finished in 16706 seconds, throughput 9144 KBytes/sec
[... Hang ...]
load: 0.18 cmd: dump 10105 [sbwait] 72.53u 383.14s 0% 73048k
load: 0.19 cmd: dump 10102 [sbwait] 164.93u 314.87s 0% 75008k
load: 0.10 cmd: dump 10102 [running] 164.93u 314.87s 0% 75008k
The timestamp on the output file is, indeed, 10:38 and the dumping
process is hanging ever since then (over 90 minutes already).
Yours,
Mikhail T.
> can you try splitting it in 2, ie no pipe?
> dump a0f some.file /old (or dump 0f - /old | gzip -c > file.dump.gz)
> restore rf some.file
>
>
restore -rf ibmo.0.2009-03-24.dump
load: 0.55 cmd: restore 11303 [nbufkv] 3.53u 3.91s 4% 27980k
unknown tape header type 213474529
abort? [yn]
Please, advise. Thanks! Yours,
Andrew Snow
> dump 0aCf 64 /ibm/ibmo.0.2009-03-24.dump /old
> DUMP: WARNING: should use -L when dumping live read-write filesystems!
I thought you said it was a read-only filesystem?
In my experience, restore can sometimes throw warnings if you dump a
live filesystem. It might be causing your errors? If possible, can you
try completely unmounting the filesystem you are dumping and trying again?
Daniel O'Connor
> Daniel O'Connor написав(ла):
> >> morning is still hanging (in sbwait) -- I've never seen this before. I'm
> >> also very troubled, that such an important functionality (dump/restore!)
> >> is sooo problem-prone, and yet so few people seem to care...
> >
> > Well, "works for me".
>
> Well, would like a login on this system to take a look for yourself? I
> can reproduce the problem easily.
I don't know the internals of dump/restore :(
> >> Is the official view, that dump is obsolete (and already bit-rotten),
> >> perhaps, and use of tar is encouraged instead?
> >
> > I've never had dump fail but it IS rather crusty and slow.. That said tar
> > doesn't cover all the information I believe.
>
> So, if dump/restore ain't it, does FreeBSD have a supported way of
> making filesystem-level backups, that's both modern and covers all
> aspects (like flags)?
I would try a pax archive, eg..
tar --format pax --one-file-system -pcf - -C / . | tar -pxf - -C /mnt/newdisk
According to the libarchive-formats page this handles ACLs & flags, my testing
shows it handles flags (at least). eg..
[midget 13:30] /tmp/test2 >touch foo
[midget 13:30] /tmp/test2 >chflags uchg foo
[midget 13:30] /tmp/test2 >tar --format pax -zpcf /tmp/test.pax.gz foo
[midget 13:30] /tmp/test2 >rm -f foo
rm: foo: Operation not permitted
[midget 13:30] /tmp/test2 >chflags nouchg foo
[midget 13:30] /tmp/test2 >rm foo
[midget 13:30] /tmp/test2 >tar -pxf /tmp/test.pax.gz
[midget 13:30] /tmp/test2 >ls -lao
total 30
drwxr-xr-x 2 darius wheel - 512 Mar 25 13:30 .
drwxrwxrwt 53 root wheel - 28672 Mar 25 13:29 ..
-rw-r--r-- 1 darius wheel uchg 0 Mar 25 13:30 foo
[midget 13:30] /tmp/test2 >rm -f foo
rm: foo: Operation not permitted
> That said, I point out, that for me, dump is not failing (although it
> did hang this morning). It is the restore, which fails to read dump's
> output:
You can't tell the difference between dump producing mangled output or restore
bombing out on valid input..
--
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
-- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
Mikhail T.
> On Wednesday 25 March 2009 13:10:23 Mikhail T. wrote:
>
>> Now can one get /real/ support for the most basic functionality of the
>> most advanced modern Unix in the world? Thanks,
>>
>
> Maybe you should return it to the shop and ask for your money back.
>
Well, if this response is the best one can get, may be I should also
revoke my own 15 years worth of contributions to the project...
Except, why would I? I always supported people, who had problems with
any of my work -- and the attitude of the rest of the contributors /used
to be/ the same...
-mi
Daniel O'Connor
> Daniel O'Connor написав(ла):
> > On Wednesday 25 March 2009 13:10:23 Mikhail T. wrote:
> >> Now can one get /real/ support for the most basic functionality of the
> >> most advanced modern Unix in the world? Thanks,
> >
> > Maybe you should return it to the shop and ask for your money back.
>
> Well, if this response is the best one can get, may be I should also
> revoke my own 15 years worth of contributions to the project...
>
> Except, why would I? I always supported people, who had problems with
> any of my work -- and the attitude of the rest of the contributors /used
> to be/ the same...
People ARE helping you, just because they haven't come up with an answer is no
reason to send snarky comments to the list.
Outrage (fake or otherwise) that people don't seem to be taking your
particular problem seriously is unhelpful and probably counter-productive.
Mikhail T.
> People ARE helping you, just because they haven't come up with an answer is no
> reason to send snarky comments to the list.
>
suggestion to eliminate the -a switch (a no-op, in fact) was
particularly unhelpful -- and deserving of mockery. Later on someone
with a similar problem will find this thread with a search engine and
will be trying to follow the posted advices -- to no avail, of course,
plunging FreeBSD further into disrepute.
> Outrage (fake or otherwise) that people don't seem to be taking your
> particular problem seriously is unhelpful and probably counter-productive.
whatever reason. Thus any talk of "productivity" is moot. Thanks for trying.
Mikhail T.
> I think before this goes any further, you will need to try
> rebooting/unmouting it, running fsck on it, and then dump the unmounted
> partition and see how that goes.
>
>
Rebooted, reran `fsck -y /old' (all clean). Same problem...
Mark Kirkwood
>
> That said, I point out, that for me, dump is not failing (although it
> did hang this morning). It is the restore, which fails to read dump's
> output:
>
> resync restore, skipped 502 blocks
> expected next file 54, got 0
> unknown tape header type -954356454
> abort? [yn] n
> resync restore, skipped 29 blocks
> expected next file 54, got 0
> unknown tape header type -1754938223
> abort? [yn] n
> resync restore, skipped 482 blocks
> expected next file 54, got 0
> unknown tape header type -915868704
> abort? [yn] n
> resync restore, skipped 29 blocks
> expected next file 54, got 0
> unknown tape header type 1790084751
> abort? [yn] n
> resync restore, skipped 482 blocks
> expected next file 54, got 0
> unknown tape header type 903667267
> abort? [yn] n
> ...
>
Is this link any help?
http://www.mail-archive.com/freebsd-...@freebsd.org/msg197899.html
Other than that, I'd suggest checking the disk(s) with smartmontools to
try to rule out hardware problems.
regards
Mark
Greg Black
> That's true. I just wanted to point out, that someone running dump only
> (to make backups) is not going to know, whether his dumps are usable
> (for whichever of the two reasons), until he needs them...
Such a person is not making backups and deserves what he gets.
I haven't got anything to say about dump/restore because I haven't
bothered with them for years. I do know that dumps from mounted file
systems will often appear to work, but will fail when it matters. This
is not a bug and is expected behaviour to which the solution is obvious.
Daniel O'Connor
> > Yes, dump is broken for you, deal with it. It is quite possible your FS
> > is corrupt, and/or your disk is damaged.
>
> ..and/or it is some other hardware problem, maybe you also should test
> your memory with memtest or something similiar? I'm using dump/restore
> very frequently and I had never seen such problem. Neither on -RELAESE,
> -STABLE, nor -CURRENT.
> So I think you should make sure that your problem is not
> hardware/filesystem dependent before you point dump/restore as a couse
> of the problem. Peter Jeremy already gives you good hints to do that.
One other thing would be to make absolutely sure that your version of dump &
restore are in sync, the are very machine/version dependent.
Peggy Wilkins
>> I've been watching this thread with some interest since we've had some
>> similar problems with dump/restore which we use every morning via cron
>> scripts on a number of servers to produce bootable clones as part of our
>> backup program. Have been doing this for years and also never saw a problem
>> as most of you say. We prefer dump/restore for backups.
>> However, last month upon upon upgrading those servers from FBSD-6.3px
>> (RELEASE) to 7.0px (RELEASE) we found that about one-half of the servers
>> had a similar problem as the original poster while the other half did not.
>> All of the servers (rackmounts) use the same (type) hardware. We spent many
>> hours trying to solve the problem with those that failed to dump/restore.
>> Also, searched for any others with the problem and only found a very few,
>> but without solutions to this issue. (Indeed, the only one was a reference
>> to any efforts to restore an older OS version which didn't apply here).
[snip]
>> SOLUTION
>> The "clones" are a very important pasrt of our backup program. Since the
>> dump side of the problems simply stuck and provided no error message at all
>> and the errors from any restores were not useful, our only solution was to
>> revert back to FBSD-6.3 on those servers with this issue and dump/restore
>> went back to working again. We left those that were working on FBSD-7.0-R
>> and they continue to work okay.
I was seeing this same problem on all my 64-bit systems: FreeBSD-7
dump would hang at a random point. Dump continues to work flawlessly
for me on FreeBSD-7/i386.
I ran across this which includes a patch:
http://www.freebsd.org/cgi/query-pr.cgi?pr=121684
The kernel patch linked to there solved the problem for me, but I am
running many production systems and am unwilling to apply this patch
to -RELEASE every time there is a kernel update (I just use the
standard GENERIC kernel which I get via freebsd-update). I now live
without dump on amd64. Apparently this fix is waiting on some related
issue; and I will be very happy when it makes it to the officially
released kernel.
plw
Hywel Mallett
> On 2009-03-24 07:30, Mikhail T. wrote:
>> dump a0f - /old | restore -rf -
>> [...]
>> DUMP: 17.25% done, finished in 3:27 at Tue Mar 24 05:42:00 2009
>> DUMP: 20.36% done, finished in 3:09 at Tue Mar 24 05:28:13 2009
>> DUMP: 23.83% done, finished in 2:50 at Tue Mar 24 05:14:32 2009
>> unknown tape header type -621260722
>> abort? [yn]
>>
>> Looks like a junk value somewhere... Unitialized variable or some such.
Looking at the restore code (tape.c, in findinode), restore is
expecting a header type in the range 1-6, so the header type
-621260722 is way out. Assuming that findinode is being passed the
correct variable, it would indicate that dump is writing the header
(or at least the header type incorrectly). I can't work out where this
header is getting written though. It looks like plenty of data gets
written into a header (such as inode, magic number, checksum). I
wonder if one of these values is overflowing and overwriting the
header type?
>
> Maybe the dump output gets corrupted in some way? (E.g. faulty RAM, or
> disk?) If you are dumping a live filesystem, could it possibly help to
> add the -L option?
It might be worth fscking the original volume (though I suspect the OP
has done this already), and also passing the -D option to restore, as
restore will then try and continue, rather than abort on getting the
invalid header type. Fixing the root cause would be better, but that
might be a workaround for now.
Peter Jeremy
>However, last month upon upon upgrading those servers from FBSD-6.3px
>(RELEASE) to 7.0px (RELEASE) we found that about one-half of the servers
>had a similar problem as the original poster while the other half did not.
>the next day, and the next day, and the next day. (The servers that did
>work fooled us and we found out about this issue on the others when the
>overlaps appeared and drew our attention). That's when our work to try and
>solve the issues started and went on for days.
It's not clear to me whether all your servers have the problem and you
only initially noticed it on some of them or some of your servers work
and others dont. In the latter case, you are probably in a very good
position to identify the problem since it is related to some difference
between your servers.
>We could only conclude that the problem was perhaps something with
>hardeware, perhaps the way memory was handled in 7.0, but that is only a
>guess.
If you are talking about server-grade hardware (ECC RAM etc) then it's
unlikely to be RAM corruption. About the only thing I can think of
would be that if you have RAM above 4GB, you might be running foul of
an address being truncated somewhere (particularly in a device
driver). (The amd64 user memory map changed between 7.x and 8.x but I
don't think there was any change between 6.x & 7.x).
The pre-emption changes in 7.x and/or moving to an SMP host would seem
to increase the probability of hitting the problem fixed in the patch
mentioned later (kern/121684).
--
Peter Jeremy
Daniel O'Connor
> On 2009-Mar-25 19:25:28 +1030, Daniel O'Connor <doco...@gsoft.com.au>
wrote:
> >One other thing would be to make absolutely sure that your version of dump
> > & restore are in sync, the are very machine/version dependent.
>
> fairly important requirement - you don't want to suddenly be unable to
> restore your backups after an upgrade). I can restore dumps made on
> FreeBSD4.8/i386 and FreeBSD4.9/alpha on a FreeBSD-current/amd64 system
> without problems.
Hmm interesting.. I must confess I haven't had problems recently but I tend to
use tar these days anyway since it's more portable.
> It's possible that you might have problems with a backup made using a
> very recent dump on a very old restore if you've used filesystem
> features that didn't exist when that restore was built.
Could be, too long ago to remember :)
Jack L. Stone
>>
>>Thanks for the reply. Forgot to mention, our machines are all i386 with the
>>problem -- so are the ones without the problem.
>>
>>Yes, I found that patch too and tried it on one of the servers -- no joy.
>>
>>Guess we'll continue to wait also for now. Maybe 7.2/i386....or, until
>>someone finds the solution since we're out of ideas and stuck with 6.3 in
>>order to use dump that we have trusted.
>
>
>Hi,
> I didnt see it in the thread clearly, but did you try
>creating a dump without the -L... i.e. without snapshots enabled ?
>
> ---Mike
>
Yes, but it's for running a dump on a (L)ive FS and just spits out warnings
to that effect and has no effect on solving the problem(s).
Jack
(^_^)
Happy trails,
Jack L. Stone
System Admin
Sage-american
Christian Walther
I followed this thread out of interest since I do not suffer from this error.
But I wonder if truss could shed some light into this issue. If for
example a dump hangs at 99%, it might be an idea to set up truss to
trace the dump process. Yes, this will produce lots of output, but
maybe it gives a hint as soon as dump hangs.
Christian Walther
Mike Tancsa
>Yes, but it's for running a dump on a (L)ive FS and just spits out warnings
>to that effect and has no effect on solving the problem(s).
Unless the filesystem is very busy, you will get your data backed up.
If you have things like databases, I still would not trust
snapshots. Better to use pg_dump or mysqldump or the app that comes
with whatever DB you are using... When backing up things like / and
/usr, I would hazard a guess that most things are not changing while
the backup is running, at least they dont in my environments. I have
never had a problem with things like /home and even /var or /mail
which are changing quite a bit. We dont restore much in the course
of our daily routine, but we have always been able to restore
people's Maildir when they accidentally have deleted stuff and it all
worked without issue over the years.
---Mike
Peter Schuller
> >to that effect and has no effect on solving the problem(s).
>
> If you have things like databases, I still would not trust
> snapshots.
Uh. If backuping up a live database from a snapshot is not
trustworthy, either the snapshot facility is broken or the database is
broken (i.e., not crash-safe to begin with).
That said there are plenty of other reasos to use proper dump tools
(data portability, confirming the ability to actually read all rows
from a table, using a more often exercised code path and perhaps less
likely to have edge case bugs, etc).
--
/ Peter Schuller
PGP userID: 0xE9758B7D or 'Peter Schuller <peter.s...@infidyne.com>'
Key retrieval: Send an E-Mail to getp...@scode.org
E-Mail: peter.s...@infidyne.com Web: http://www.scode.org
Mike Tancsa
> >
> > Unless the filesystem is very busy, you will get your data backed up.
> > If you have things like databases, I still would not trust
> > snapshots.
>
>Uh. If backuping up a live database from a snapshot is not
>trustworthy, either the snapshot facility is broken or the database is
>broken (i.e., not crash-safe to begin with).
... or the database is configured in a risky way... But judging by
this thread, it seems dump with -L is indeed broken for some
people. Hence, I suggested dumping the database using the database's
backup tools and trying dump without -L
---Mike
Peter Schuller
> where the database and transaction log files are DIFFERS. You can get
> into a pathological case in that instance.
>
> If the transaction log and database itself are both on the same
> snapshotted entity (that is, the snapshot is pulled at the same instant
> in time for both) what you get BETTER be restorable or your database's
> transaction log facility doesn't really do what it promises to do!
Absolutely. Doing things like snapshot based backups of databases
assumes you know what you're doing since it is not something which is
documented as an official procedure in your typical database
administrator guide.
Personally, while I would use such schemes, I would always use a plain
fully supported regular dump as a fallback position. I would only rely
on snapshot based processes to do fancy stuff (such as near-realtime
hot standby with zfs snaps + serialized incrementals).
Karl Denninger
>> To add to this what SHOULD (ha!) work is to dump the database partition
>> FIRST and THEN dump the Transaction Log partition.
>>
>
> Depending on how the database works; specifically when old data in the
> log may or may not be expunged. You could do this with PostgreSQL by
> using it's PITR support for example. But to the extent that you have a
> log which is only supposed to be used for internal reasons (such as
> with pg by default), you'd likely be in trouble anyway unless you had
> a specific reason to know that it is safe.
>
True. If the log rolls on you while you're taking the dumps you're DEAD.
Both design and verification are important :)
--
--
Karl Denninger
ka...@denninger.net
Karl Denninger
>
>> That said there are plenty of other reasos to use proper dump tools
>> (data portability, confirming the ability to actually read all rows
>> from a table, using a more often exercised code path and perhaps less
>> likely to have edge case bugs, etc).
>
> database to perform the backup. Most commercial DBs have hooks that
> allow the backup routines to call out to custom snapshot facilities.
> One would usually request a backup through the database, which would
> then freeze IO to its data files and maybe log files, deal with
> flushing caches etc and then call your snapshot routine. I'm not
> aware that MySQL and Postgres do though so the best you can do is a dump.
>
> Jake
VERY careful thought has to go into backup strategy with production
databases.
Hooks that call out and snapshot are not necessarily good enough
although they're "necessary" to get a dump that restores without the
database going into log-replay mode.
It is not difficult to do this with Postgresql; you can quiesce the
database, snapshot and then release it, then dump the snapshots. This
gives you transaction-complete dumps (as opposed to "crashed and rolled
forward" dumps). The latter ("crashed and rolled forward"), if its
sufficient, is trivially able to be done by having Postgresql (and most
other databases) keep a sufficient number of log segments that a
rollover cannot happen during the dump process itself, and either
snapshotting both filesystems at once, keeping both on the same
filesystem (undesirable for performance reasons) or dumping the database
first and XLOG second.
However, whether either of these approaches is sufficient is another
matter. One of the real problems with live transaction processing
systems is a means to know when there is a failure exactly what you
lost. This is not a trivial problem to solve and requires plenty of
thought before implementation, especially if you cannot afford the
outage time necessary to take the snapshots - in some cases even that
(relatively) short outage time is unacceptable.
David Malone
> Same problem:
>
> restore -rf ibmo.0.2009-03-24.dump
> load: 0.55 cmd: restore 11303 [nbufkv] 3.53u 3.91s 4% 27980k
> unknown tape header type 213474529
> abort? [yn]
>
> Please, advise. Thanks! Yours,
Hi Mikhail,
If you actually need to get a dump back that restore can't read,
you can try the "-D" option that I added a few years ago. Dump and
restore expect things to be in a block format, but if (say) dump
outputs a few bytes into the stream due to a bug, then the entire
end of the dump can become unreadable. The -D option to restore
tells it to try hard to get back in sync again.
I'd guess you've tripped over either a bug in dump or restore. If
you can file a PR, particularly with access to a sample dump, then
I can have a look and see if I can figure out what's going on.
Daid.
Pete French
> to perform the backup. Most commercial DBs have hooks that allow the
> backup routines to call out to custom snapshot facilities. One would
> usually request a backup through the database, which would then freeze IO
> to its data files and maybe log files, deal with flushing caches etc and
> then call your snapshot routine. I'm not aware that MySQL and Postgres do
> though so the best you can do is a dump.
mysql can do this - you can flush the tables and acuire a lock
simultaneously so that you can then snapshot the uderlying filesystem
and then release the lock to let everything continue. I use this for taking
database snapshots and it works fine. I stop my slaves before snapshotting
to avoid log files changing underneath me too .... like this...
#!/bin/sh
/usr/local/bin/mysql -usnapuser -psnapuser <<EOF
slave stop;
flush tables with read lock;
system /sbin/zfs snapshot archive/mysql@latest;
unlock tables;
slave start;
EOF
That appears to work fine. I do also do other dumps, but the above
works nicely for a quick and easy snapshot on a slave which can be
rolled back in the case of a crash (and will then update from the master
properly)
-pete.
Jack L. Stone
>> ... or the database is configured in a risky way... But judging by
>> this thread, it seems dump with -L is indeed broken for some
>> people. Hence, I suggested dumping the database using the database's
>> backup tools and trying dump without -L
>
>unless you absolutely have to. That is assuming snapshots are not
>broken so that you can still use mksnap_ffs with your favorite backup
>tool.
>
>--
>/ Peter Schuller
>
No one has said the dump "L" is broken -- and is NOT but now may mislead
others just tuning into this thread.
Jack
(^_^)
Happy trails,
Jack L. Stone
System Admin
Sage-american
Tom Evans
..
> Absolutely. You really must use a tool that interacts with the database
> to perform the backup. Most commercial DBs have hooks that allow the
> backup routines to call out to custom snapshot facilities. One would
> usually request a backup through the database, which would then freeze IO
> to its data files and maybe log files, deal with flushing caches etc and
> then call your snapshot routine. I'm not aware that MySQL and Postgres do
> though so the best you can do is a dump.
>
>
>
Just to add, mysql has a utility (mysqlhotcopy) to allow you to directly
copy MyISAM databases with a guarentee of consistency (thus avoiding the
conversion from MyISAM data -> SQL, and no need to reimport when
recovering). It isn't exactly online though, any writes will be blocked
until the hotcopy finishes.
Still, it is only MyISAM, and not much call for that these days..
Cheers
Tom
Karl Denninger
> At 01:11 PM 3/26/2009, Jack L. Stone wrote:
>
>> No one has said the dump "L" is broken -- and is NOT but now may mislead
>> others just tuning into this thread.
>
>
> -L....
> I wonder if the original poster just needs to do an fsck on the disk.
> The only time I had dump crap out on me (other than with -L) was on a
> dirty filesystem.
>
> I regularly do stuff like
> cd /usr;dump 0f - /usr | (cd /tmp/usr; restore -rf - )
> to duplicate partitions
>
> ---Mike
I do as well but if the original user is trying to restore into a
filesystem that is either not clean or not EMPTY for a full restore he
could run into some trouble.
I've never had dump/restore fail by the way, other than due to bad
(tape) media (yes, it was that long ago :) )
Eugene Gladchenko
MT> I'm trying to migrate a filesystem from one disk to another using:
MT> dump a0hCf 0 32 - /old | restore -rf -
MT> (/old is already mounted read-only). The process runs for a while and
MT> then stops with:
MT> [...]
MT> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009
MT> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009
MT> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009
MT> unknown tape header type 1853384566
MT> abort? [yn]
MT> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's
MT> dump's output?
Is your /tmp large enough to restore?
--
Eugene Gladchenko
EVG15-RIPE
Rolf G Nielsen
> Hello!
>
> I'm trying to migrate a filesystem from one disk to another using:
>
Try
dump -a0 -h 0 -C 32 -f - /old
instead.
Your command line should work, but I've had trouble with similar ones
and have been able to solve it by separating the options.
>
> (/old is already mounted read-only). The process runs for a while and
> then stops with:
>
> [...]
> DUMP: 22.85% done, finished in 3:57 at Tue Mar 24 01:03:21 2009
> DUMP: 24.66% done, finished in 3:50 at Tue Mar 24 01:00:58 2009
> DUMP: 26.44% done, finished in 3:43 at Tue Mar 24 00:59:14 2009
> unknown tape header type 1853384566
> abort? [yn]
>
> Any idea, what's going on? Why can't FreeBSD's restore read FreeBSD's
> dump's output?
>
> The system runs 7.0-STABLE from July 6th, i386. I just tried updating
> the dump and restore from source (latest 7.x) -- the error is the same...
>
> Please, advise. Thanks!
>
> -mi
>
> _______________________________________________
> freebsd...@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stabl...@freebsd.org"
>
>
>
--
Sincerly,
Rolf Nielsen
Mikhail T.
> Sorry, this person is *not* making backups in any meaningful fashion.
> Unless you verify regularly (preferably every time you make a backup)
> that you can restore both parts of the backup and the entire thing, you
> are not making backups.
To qualify for your (and your kind's) recognition then, a person needs
to have at least as much extra storage capacity as the largest
filesystem they are backing up. They also need non-trivial scripting
abilities, because the OS doesn't include anything like what you are
describing (and I already do consider scheduling dumps via cron
"trivial", which may be a stretch). Yours may thus be an acceptable
requirement for a multi-computer shop with dedicated system
administration personnel, but for a private home user with only one
computer this simply is not reasonable.
Stating this as a requirement is ridiculous -- unless you are prepared
to say, that such people should not own a computer (with worthy data) at
all. And that's even more ridiculous... Make your pick.
I would agree with you, if the chosen backup method involved some
complex or third-party tools. But if the simple, OS-supplied orthogonal
dump/restore don't work together, then the OS is broken -- plain and
simple, and pointing a finger at the user: "Well, it is all your fault,
because you relied on us providing you with working utilities,
ha-ha-ha!" -- is the lamest excuse imaginable.
-mi
P.S. Some people have actually volunteered to help debug this problem
and I'm working on providing them with data (the troublesome partition
is, sadly, over 170Gb, so it takes a while). Any results/conclusions
will be posted under the original subject.
P.P.S. The data transferred fine using tar, but that is not the point --
the bug (confirmed by at least one more person) -- needs to be fixed
before a higher-profile embarrassment...
Andrew Snow
> To qualify for your (and your kind's) recognition then, a person
> needs to have at least as much extra storage capacity as the
> largest filesystem they are backing up. They also need
> non-trivial scripting abilities, because the OS doesn't
> include anything like what you are describing
Mikhail, users would be well advised to check their backups using this
option, without having to have the space to restore:
-N Do the extraction normally, but do not actually write any
changes to disk. This can be used to check the integrity of dump media
or other test purposes.
RESTORE(8) FreeBSD System Manager's Manual