[gentoo-user] is ddrescue this slow?
Valmor de Almeida
Hello,
I am trying to recover data from a failed drive. An initial attempt with dd took over 12 hours and it was not even at half of the 80GB damaged drive; so I quit. Info on the web pointed me to ddrescue
ddrescue -n /dev/sda /dev/sdc rescued.log
which has taken over 6 hours so far
Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 0 B, errsize: 0 B, errors: 0
Current status
rescued: 58656 MB, errsize: 4408 kB, current rate: 4096 B/s
ipos: 58660 MB, errors: 32, average rate: 2958 kB/s
opos: 58660 MB, time from last successful read: 0 s
Copying non-tried blocks...
The "current rate" varies and sometimes is down to a few B/s!
Is this normal? The drive copied to is a USB external drive.
Thanks for inputs.
--
Valmor
Valmor de Almeida
Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 0 B, errsize: 0 B, errors: 0
Current status
ipos: 58786 MB, errors: 66, average rate: 1751 kB/s
opos: 58786 MB, time from last successful read: 0 s
Copying non-tried blocks...
Sometimes the "current rate" reads 0 B/s for a long time... and "time from last successful read" can be 8m.
Would any one know whether this is normal?
Thanks,
--
Valmor
Neil Bothwick
> Sometimes the "current rate" reads 0 B/s for a long time... and "time
> from last successful read" can be 8m.
>
> Would any one know whether this is normal?
Doesn't ddrescue retry on blocks it cannot read? That would explain the
variable read rate, even the period of zero activity. If your drive is
that badly damaged, dd would have been no use anyway.
--
Neil Bothwick
There's too much blood in my caffeine system.
Stroller
On 9 Jan 2010, at 09:23, Neil Bothwick wrote:
> On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
>
>> Sometimes the "current rate" reads 0 B/s for a long time... and "time
>> from last successful read" can be 8m.
>>
>> Would any one know whether this is normal?
>
> Doesn't ddrescue retry on blocks it cannot read? That would explain
> the
> variable read rate, even the period of zero activity. If your drive is
> that badly damaged, dd would have been no use anyway.
I think Valmor is using GNU ddrescue, with which one makes the
multiple passes manually. The "-n" flag on the command line that
Valmor posted (`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates to
the examples given in the GNU manual page [1]. I believe that GNU
ddrescue is the better version - it was inspired by garloff's original
work, and makes improvements, but it operates differently.
Having said that, it could just be that the drive _firmware) is making
multiple attempts to read the failing blocks before returning the
failure (or the data, in the case that a 2nd attempt to read the drive
was successful) to the host o/s. Isn't this how hard-drives work?
ddrescue worked fast here when I tried it here recently on a drive
with only one unreadable block, but Valmor's drive is failing much
more severely. TBH, I would expect reads from a badly-failing drive,
but this is an intuitive expectation, not a reasoned one. I think the
best thing he can do is hold his breath, wait until its finished and
see how if the results are readable, after running `fsck` on the
mounted filesystem.
Valmor: when I ran the `ddrescue -dr3` stage I had no success at all,
however the system was fine after a reboot & a `chkdsk`. Better than
it had been, in fact, on the old hard-drive. You might have more luck
getting *some* of the blocks showing as failed when you run it on your
drive, but don't be too disheartened if you don't.
Stroller.
[1] http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Examples
Grant Edwards
> On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
>
>> Sometimes the "current rate" reads 0 B/s for a long time... and "time
>> from last successful read" can be 8m.
>>
>> Would any one know whether this is normal?
>
> Doesn't ddrescue retry on blocks it cannot read?
>
> That would explain the variable read rate, even the period of
> zero activity. If your drive is that badly damaged, dd would
> have been no use anyway.
Yes. On a disk with a lot of bad blocks it can be very slow as
it retries bad blocks and narrows the failed regions.
--
Grant
Valmor de Almeida
[snip]
in the GNU manual page [1]. I believe that GNU ddrescue is the better version - it was inspired by garloff's original work, and makes improvements, but it operates differently.
Comment. Another reason I moved away from dd (apart from the slow running time) to ddrescue was because of this note related to LVM.
http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html
"Steve Holmes reports that dd with conv=sync,noerror doesn't correctly image disks with LVM2 Logical Volumes. I haven't investigated this. He also points out GNU ddrescue ( not the same as dd_rescue mentioned above) which looks useful. According to Steve, ddrescue works finewith LVM2, and some people seem to suggest it's generally superior to dd_rescue."
The partition I would like to get data from is under LVM (previous post).
Thanks,
--
Valmor
Valmor de Almeida
I think Valmor is using GNU ddrescue, with which one makes the multiple passes manually. The "-n" flag on the command line that Valmor posted (`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates to the examples given in the GNU manual page [1]. I believe that GNU ddrescue is the better version - it was inspired by garloff's original work, and makes improvements, but it operates differently.
On 9 Jan 2010, at 09:23, Neil Bothwick wrote:
On Sat, 9 Jan 2010 07:20:18 +0000, Valmor de Almeida wrote:
Sometimes the "current rate" reads 0 B/s for a long time... and "time
from last successful read" can be 8m.
Would any one know whether this is normal?
Doesn't ddrescue retry on blocks it cannot read? That would explain the
variable read rate, even the period of zero activity. If your drive is
that badly damaged, dd would have been no use anyway.
Indeed I am using GNU ddrescue and the -n flag is supposed to expedite the recovery of data as posted in http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
"The best solution - both faster and more efficient - seems to be Antonio Diaz's 'ddrescue'
(ddrescue)"
# first, grab most of the error-free areas in a hurry:
./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
# then try to recover as much of the dicy areas as possible:
./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log
expectation, not a reasoned one. I think the best thing he can do is hold his breath, wait until its finished and see how if the results are readable, after running `fsck` on the mounted filesystem.
The first step above finished; don't know how long it took but it was a long time (maybe 20 hours or more?) and the screen output was
Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 0 B, errsize: 0 B, errors: 0
Current status
ipos: 58860 MB, errors: 95, average rate: 1365 kB/s
opos: 58860 MB, time from last successful read: 0 s
Copying non-tried blocks...
ddrescue: write error: Input/output error
Comparing with the screen output at the time of my first post,
Current status rescued went from 58656 MB to 58811 MB, errsize went from 4408 kB to 48909 kB.
Don't know how the write error: Input/output error message affect the data in the new drive copied to. Not sure whether I should do the next step with option -r 1.
This failed drive is still bootable and the corruption is in the partitions /var (which I do not care) and /home; these cannot be mounted. I would like to attempt to get a couple of files from /home that were not in the most recent backup. Maybe I should try to rescue only the partition /home. However this partition is under LVM. Specifically, /dev/sda4 is a linux LVM partition. The volume group is vfda and the logical volume of interest is /dev/vfda/home which has reiserfs file system. Is it possible to rescue data only from this partition when under LVM?
Valmor: when I ran the `ddrescue -dr3` stage I had no success at all, however the system was fine after a reboot & a `chkdsk`. Better than it had been, in fact, on the old hard-drive. You might have more luck getting *some* of the blocks showing as failed when you run it on your drive, but don't be too disheartened if you don't.
Stroller.
Stroller, you mean your rescue.log showed no problematic entries? I got over 400 lines in my rescue.log file.
root@sysresccd /root % head rescued.log
# Rescue Logfile. Created by GNU ddrescue version 1.11
# current_pos current_status
0xDB45D9000 ?
# pos size status
0x00000000 0x9CE341000 +
0x9CE341000 0x00000200 -
0x9CE341200 0x0001F000 *
0x9CE360200 0x00000200 -
0x9CE360400 0x00020000 *
0x9CE380400 0x3BD63AC00 +
Stroller
Dale
Dale
:-) :-)
Stroller wrote:
> Would love to comment on this. Is it possible you could resend this
> post in plain text format?
>
> Stroller.
>
>
> On 10 Jan 2010, at 02:08, Valmor de Almeida wrote:
>
>>
>> Indeed I am using GNU ddrescue and the -n flag is supposed to
>> expedite the recovery of data as posted in
>> http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
>>
>> "The best solution - both faster and more efficient - seems to be
>> Antonio Diaz's 'ddrescue' (ddrescue
>> <http://savannah.gnu.org/projects/ddrescue/>)"
Valmor de Almeida
Would love to comment on this. Is it possible you could resend this post in plain text format?Stroller.
Below is my last post copied and pasted into gmail without the html hot links. I am doing this from within systemrescuecd using firefox and gmail. Don't know how to make it plain ascii otherwise. Hopefully just eliminating the html links will work. Stroller, is this what you are referring to?
Thanks,
--
Valmor
On Sat, Jan 9, 2010 at 11:10 AM, Stroller wrote:
[snip]
in the GNU manual page [1]. I believe that GNU ddrescue is the better version - it was inspired by garloff's original work, and makes improvements, but it operates differently.
Comment. Another reason I moved away from dd (apart from the slow running time) to ddrescue was because of this note related to LVM.
h
Stroller
On 10 Jan 2010, at 21:28, Stroller wrote:
> On 10 Jan 2010, at 18:09, Valmor de Almeida wrote:
>> ... I am doing this from within systemrescuecd using firefox and
>> just eliminating the html links will work. Stroller, is this what
>> you are referring to?
>
> to access your gmail account. You're using Gentoo, so there are
> plenty of clients which enable you to do this: mutt, pine,
> Thunderbird, Kmail, ... any of them can post in plain text.
I just reread the above ("from within systemrescuecd"), which implies
you may be stuck without another working system.
Google says "If you decide you'd like to write a message in plain text
format, just click << Plain text along the top of the compose window."
<http://mail.google.com/support/bin/answer.py?hl=en&answer=8260>
If you're stuck without another working system and this doesn't work
then let me know & I'll make the extra effort, with apologies (and
with sympathies for your difficult working conditions - I am sorry if
I have exacerbated them).
Stroller.
Stroller
On 10 Jan 2010, at 18:09, Valmor de Almeida wrote:
> On Sun, Jan 10, 2010 at 8:09 AM, Stroller <stro...@stellar.eclipse.co.uk
> > wrote:
>> Would love to comment on this. Is it possible you could resend this
>> post in plain text format?
>>
>> Stroller.
>
>
> Below is my last post copied and pasted into gmail without the html
> hot links. I am doing this from within systemrescuecd using firefox
> and gmail. Don't know how to make it plain ascii otherwise.
> Hopefully just eliminating the html links will work. Stroller, is
> this what you are referring to?
Both messages contain html text formatting. If you look at your last
message (the one to which I'm replying now, Message-ID: <128ccc221001101009v75f...@mail.gmail.com
>) in a text editor, for instance:
<div><div></div>
<div class=3D"h5"><div><br></div></div></div></div></
blockquote><div><br>Be=
low is my last post copied and pasted into gmail without the html hot
links=
. I am doing this from within systemrescuecd using firefox and gmail.
Don&#=
39;t know how to make it plain ascii otherwise. Hopefully just
eliminating =
the html links will work. Stroller, is this what you are referring to?
<br>
<br>Thanks,<br><br>--<br>Valmor<br>=A0<br></div><blockquote
class=3D"gmail_=
quote" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt
0pt =
0pt 0.8ex; padding-left: 1ex;"><div style=3D"word-wrap: break-
word;"><div><=
div class=3D"h5">
<div></div><br><div class=3D"im">On Sat, Jan 9, 2010 at 11:10 AM,
Stroller=
=A0<span dir=3D"ltr"></span> wrote:<br></div><div
class=3D"gmail_quote"><di=
v><br>[snip]<br>=A0<br></div><div class=3D"im"><blockquote
class=3D"gmail_q=
uote" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt
0pt 0=
pt 0.8ex; padding-left: 1ex;">
The problem I have with this is that it seems difficult in my mail
client to snip excessive quoting when I reply. If I switch to plain
text, all the quoting marks / indenting disappears because (as per the
snippet above) it's using an arbitrary html construction ("border-
left: 1px solid; margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> to
represent that.
Just use plain text. It's the convention. It's simple, it works. If
Gmail imposes this html upon you then I suggest you use IMAP to access
your gmail account. You're using Gentoo, so there are plenty of
clients which enable you to do this: mutt, pine, Thunderbird,
Kmail, ... any of them can post in plain text.
I'm sorry to sound like a knob - I do admit to having a bee in my
bonnet about this at the moment. I have a client who sends me 100kb
emails with tiny unreadable text, a yellow background and multiple
images (at least 7), just to convey a sentence or two of actual
content. They pad maybe 150 bytes of text with 100,000 bytes of crap
(the entire text of "Pride & Prejudice" is 700kb), and its effect is
to make the content less readable; I can't bitch to them about their
idiocy, because they pay me thousands a year.
I know this isn't your fault, but I had been quite busy for some hours
and was perhaps tired when I clicked reply to your message last night.
I clicked on the text so as to quote inline and a whole paragraph lit
up as my mailer tried to deal with it; so I clicked on y mailer's
plain text button and all the quote marks disappeared and I couldn't
see who had said what. I just couldn't be arsed to deal with it. Just
use plain text. It's the convention. It's simple, it works.
Stroller.
Valmor de Almeida
<stro...@stellar.eclipse.co.uk> wrote:
>
> On 10 Jan 2010, at 18:09, Valmor de Almeida wrote:
>
>> On Sun, Jan 10, 2010 at 8:09 AM, Stroller <stro...@stellar.eclipse.co.uk>
>> wrote:
>>>
>>> Would love to comment on this. Is it possible you could resend this post
>>> in plain text format?
>>>
>>> Stroller.
>>
>>
>> Below is my last post copied and pasted into gmail without the html hot
>
> Both messages contain html text formatting. If you look at your last message
> (the one to which I'm replying now, Message-ID:
> <128ccc221001101009v75f...@mail.gmail.com>) in a text
> editor, for instance:
I share all of your comments. Yes I was/am stuck inside
systemrescuecd. I typically use thunderbird to get my mail from the
gmail server as imap and always use plain text (to send and receive).
Therefore I am not knowledgeable of the web gmail application; I
seldom log into my gmail account with a web browser. Your next e-mail
pointed me to the "plain" option; thanks! Don't know how I could have
missed it. Originally I had looked at the settings of my gmail account
but it did not help. On my next e-mail I will send a clean plain text
resend.
Thanks,
--
Valmor
Valmor de Almeida
> plain text format?
> Stroller.
Here it goes.
>On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stro...@stellar.eclipse.co.uk> >wrote:
[snip]
> in the GNU manual page [1]. I believe that GNU ddrescue is the better >version - it was inspired by garloff's original work, and makes improvements, >but it operates differently.
Comment. Another reason I moved away from dd (apart from the slow
running time) to ddrescue was because of this note related to LVM.
http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html
Valmor de Almeida
<stro...@stellar.eclipse.co.uk> wrote:
>
[snip]
>
> I just reread the above ("from within systemrescuecd"), which implies you
> may be stuck without another working system.
Yes.
I am afraid the text below will be confusing but will send anyway.
Will be happy to send clarifications if needed. This is a resend (in
plain text) of another e-mail I sent with information on the LVM
partition on the broken drive.
>On Sat, Jan 9, 2010 at 11:10 AM, Stroller <stro...@stellar.eclipse.co.uk> >wrote:
[snip]
> I think Valmor is using GNU ddrescue, with which one makes the multiple >passes manually. The "-n" flag on the command line that Valmor posted >(`ddrescue -n /dev/sda /dev/sdc rescued.log`) relates to the examples given >in the GNU manual page [1]. I believe that GNU ddrescue is the better version >- it was inspired by garloff's original work, and makes improvements, but it >operates differently.
Indeed I am using GNU ddrescue and the -n flag is supposed to expedite
the recovery of data as posted in
http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
"The best solution - both faster and more efficient - seems to be
Antonio Diaz's 'ddrescue' (ddrescue)"
# first, grab most of the error-free areas in a hurry:
./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
# then try to recover as much of the dicy areas as possible:
./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log
> expectation, not a reasoned one. I think the best thing he can do is hold >his breath, wait until its finished and see how if the results are readable, after >running `fsck` on the mounted filesystem.
The first step above finished; don't know how long it took but it was
a long time (maybe 20 hours or more?) and the screen output was
Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 0 B, errsize: 0 B, errors: 0
Current status
rescued: 58811 MB, errsize: 48909 kB, current rate: 83 B/s
ipos: 58860 MB, errors: 95, average rate: 1365 kB/s
opos: 58860 MB, time from last successful read: 0 s
Copying non-tried blocks...
ddrescue: write error: Input/output error
Stroller
On 11 Jan 2010, at 00:14, Valmor de Almeida wrote:
> ...
> the recovery of data as posted in
> http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
>
> "The best solution - both faster and more efficient - seems to be
> Antonio Diaz's 'ddrescue' (ddrescue)"
>
> # first, grab most of the error-free areas in a hurry:
> ./ddrescue -n /dev/old_disk /dev/new_disk rescued.log
> # then try to recover as much of the dicy areas as possible:
> ./ddrescue -r 1 /dev/old_disk /dev/new_disk rescued.log
I remember reading a very similar summary - the words "grab most of
the error-free areas in a hurry" seem quite familiar - when I started
using ddrescue, a few months ago.
I would try running fsck on a copy of the image.
Your original command was to clone to /dev/sdc - you can now safely
`dd if=/dev/sdc2 of=/mnt/foo/olddrive.img` and then loopback mount the
image. You should be able to fun fsck on it, also. (Where sdc2 is the
partition /home was on)
I have invested in a fair amount of hard-drive space recently, and the
extra room allows you a lot of breathing space for operations like this.
If fsck doesn't work then you can dry photorec. I would try on both
sdcX and olddrive.img (where X is the partition /home was on).
DISCLAIMER: I don't know anything about LVM or how that might affect
things. In fact, I avoid LVM for this reason. I might do something
like backing up (using `dd`) the other partitions that contribute to
the affected volume groups, then lvm-mount them together with your
sdcX and try an reiserfsck on the resultant logical volume. The idea
position to be in (well, "ideal" once we have accepted the fact of the
failed drive) is to have images of all your drives before you start
mucking about any further; if the repair attempt fails you can revert
to the images and try again a different way.
>> Valmor: when I ran the `ddrescue -dr3` stage I had no success at
>> all, however the system was fine after a reboot & a `chkdsk`.
>> Better than it had been, in fact, on the old hard-drive. You might
>> have more luck getting *some* of the blocks showing as failed when
>> you run it on your drive, but don't be too disheartened if you don't.
>
> Stroller, you mean your rescue.log showed no problematic entries?
No, I had only one line in my rescue.log, my most recent attempt at
this. The `ddrescue -dr3` stage errored without managing to recover
that block, but it seems like the data in that block was not really
important.
The old drive failed to completely boot Windows because of this bad
block; the new drive - with that block missing - booted fine (perhaps
after a chkdsk). So I can only conclude that - using the old drive -
Windows died because some unimportant system file was thought to be
there. It couldn't read the block and kept trying. With the new drive
the file was clearly absent or damaged, so Windows was able to
continue the boot without it. Perhaps some unimportant system service
failed to start, but in any case the system worked fine.
I'm pretty sure I also dealt last year with another drive which had
about 3 errors on it, but I don't remember any of the details.
> I got over 400 lines in my rescue.log file.
>
> root@sysresccd /root % head rescued.log
> # Rescue Logfile. Created by GNU ddrescue version 1.11
> # current_pos current_status
> 0xDB45D9000 ?
> # pos size status
> 0x00000000 0x9CE341000 +
> 0x9CE341000 0x00000200 -
> 0x9CE341200 0x0001F000 *
> 0x9CE360200 0x00000200 -
> 0x9CE360400 0x00020000 *
> 0x9CE380400 0x3BD63AC00 +
It seems to me that your drive was in a very bad way, and this is why
you has so many errors. You don't say how large it was - ddrescue
terminated at 58811 MB. Was it only a 60GB drive?
It seems to me that ddrecue died before managing to complete the first
pass. I don't think you should have seen the "ddrescue: write error:
Input/output error" line if the first pass completed ok - you should
have just seen something like "Completed: XXX copied, Y errors". Then
you would try a second pass (eg `ddrescue -r 3`) and *hope* to get
fewer errors the second time around; the -r 3 tells ddrescue to keep
trying, make 3 attempts to read this block before giving up. Because
the logfile keeps a record of the failed blocks, when you run ddrescue
with the -r flag it only tries the blocks that have previously failed.
So you can quite safely keep running `ddrescue -r 3 /dev/old_disk /dev/
new_disk rescued.log`, optimistically hoping that maybe this time the
read will be successful. The only important thing is to make sure you
always use the same destination and rescued.log file when doing so -
using the same rescued.log file ensures you're not wasting time trying
blocks that you've already got on a previous pass, and using the same
destination ensures that any blocks that are read successfully this
time are written into an image file that is combined from the multiple
passes.
I think that the "write error: Input/output error" line indicates that
your drive really has failed badly & catastrophically, and that you're
not going to get any more off it. Feel free to try though. Console
yourself with the knowledge that ddrescue was the best tool for the
job, and that in using it you did all you could have done to recover
your data.
The number of errors you saw (95) indicates your drive was much worse
than the ones I have recently worked on. I'm pretty sure my
rescued.log was less than 40 lines long (I could `cat` or `less` it,
and it fitted within my 80 wide x 50 tall terminal, I'm sure).
I hope this makes sense. I'm by no means an expert, but I'm glad to
help in any way possible. Having lots of disk space helps a lot.
Stroller.
Valmor de Almeida
[snip]
>
> I would try running fsck on a copy of the image.
>
I did try and thought I would post here some final info just for the
record. I proceeeded with the command
root@sysresccd /root % ddrescue -r 1 /dev/sda /dev/sdc rescued.log
which took ~50 hours to finish with the message:
Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued: 58811 MB, errsize: 48909 kB, errors: 95
Current status
rescued: 77815 MB, errsize: 2210 MB, current rate: 0 B/s
ipos: 66589 MB, errors: 598, average rate: 56872 B/s
opos: 66589 MB, time from last successful read: 18.5 m
Trimming failed blocks...
ddrescue: write error: Input/output error
Again the same error message. I did not care about it and moved forward
to mount the /dev/sdc drive. Enabled the LVM volume groups with vgchange
-a y (all this under the systemrescuecd boot), mounted the partition of
interest under LVM control and did a
reiserfsck --check /dev/myvg/mylv
It ended with
1 found corruptions can be fixed only when running with --rebuild-tree
###########
reiserfsck finished at Fri Jan 15 11:46:23 2010
###########
The next step was then
reiserfsck --rebuild-tree --logfile rebuild.log /dev/myvg/mylv
I was then able to mount the partition and inspect the newly created
lost+found/ directory. Surprisingly I was able to find the file I was
looking for!!
It was the first time I tried this kind of HDD forensics and was
surprised with the time that it took to recover data from a relative low
storage drive: 80GB. The rebuild.log file had over 8000 lines.
Stroller, thanks for all your comments and suggestions. Yes having extra
disk space is a must to be able to recover data.
--
Valmor