Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

TestDisk,PhotoRec , or a Data Recovery Service First?

226 views
Skip to first unread message

Steve Mysterious

unread,
Jan 15, 2021, 10:32:25 PM1/15/21
to
I have an external SSD hard drive, about 1 Terabyte.

The other day I was in a rush, I was trying to format a thumb drive, and accidentally formatted the SSD hard drive instead.

I read that it is harder to recover data from an SSD hard drive due to the different technology.

I heard about TestDisk and PhotoRec.

If I try those two applications before going to a data recovery service is it possible to the situation worse in terms of recovering data?

Thanks for any opinions.

Steve

The Natural Philosopher

unread,
Jan 16, 2021, 12:09:56 AM1/16/21
to
I can think of no way to recover SSD data. I doubt it will even be
there. If a sector by sector read doesnt reveal it, its gone.

dd the whole drive onto another disk and see what you can pick out.

But I an not hopeful



--
To ban Christmas, simply give turkeys the vote.

Pascal Hambourg

unread,
Jan 16, 2021, 2:53:24 AM1/16/21
to
Le 16/01/2021 à 04:32, Steve Mysterious a écrit :
> I have an external SSD hard drive, about 1 Terabyte.
>
> The other day I was in a rush, I was trying to format a thumb drive, and accidentally formatted the SSD hard drive instead.
>
> I read that it is harder to recover data from an SSD hard drive due to the different technology.

Physically : in a rotating hard disk, overwritten data are immediately
erased when writing new data. In a SSD, overwritten data are not
immediately erased when writing new data, because new data are written
in a new physical block and the old block is just discarded and will be
erased by the garbage collector at any time.

Logically : no difference. Old overwritten data is not accessible
through the SSD standard interface.

A specific feature of SSD is TRIM/discard. If the formatting tool sends
a blkdiscard command for all blocks before formatting the device, then
these blocks may be erased by the garbage collector at any time.

> I heard about TestDisk and PhotoRec.

Testdisk is only for lost partition recovery, not data recovery.

> If I try those two applications before going to a data recovery service is it possible to the situation worse in terms of recovering data?

Photorec does not write to the source device. It can only make things
worse if the device is physically damaged. But if all blocks were
discarded, then powering the SSD gives it time to physically erase more
discarded data. However this makes a difference only if the SSD
controller still returns the original data from discarded sector read
(instead of random data or zero) or the recovery service is able to read
the physical blocks.

Steve Mysterious

unread,
Jan 16, 2021, 11:56:02 AM1/16/21
to
On Saturday, January 16, 2021 at 2:53:24 AM UTC-5, Pascal Hambourg wrote:
> Le 16/01/2021 à 04:32, Steve Mysterious a écrit :
> > I have an external SSD hard drive, about 1 Terabyte.
> >
> > The other day I was in a rush, I was trying to format a thumb drive, and accidentally formatted the SSD hard drive instead.
> >
> > I read that it is harder to recover data from an SSD hard drive due to the different technology.
> Physically : in a rotating hard disk, overwritten data are immediately
> erased when writing new data. In a SSD, overwritten data are not
> immediately erased when writing new data, because new data are written
> in a new physical block and the old block is just discarded and will be
> erased by the garbage collector at any time.

Interesting comment. I disconnected the drive as soon as I learned what happened. From what you wrote the possibility of recovering data
does not sound completely hopeless.

I just got back from taking the drive to a Microcenter service desk. They told me they outsource data recover, and if it worked the company would charge me about $1,350 USD since the drive holds a terabyte.

Is there Linux data recovery software you would recommend?

My big problem is that my computer hard drive is much smaller than 1 terabyte.

Thanks for any clues.


Carlos E.R.

unread,
Jan 16, 2021, 3:28:09 PM1/16/21
to
If you only formatted the disk there is a good chance of recovering most
of the data.

You would have to prepare a Linux system with fstrim disabled, connect
the failed disk, image it in full to a suitable big hard disk, then
disconnect the disk. From that moment, you would work on copies of this
copy.

First try would be testdisk, then photorec. Possibly others, free or
proprietary.

I don't know about tools that can access discarded blocks. Possibly
those are manufacturer tools.


--
Cheers, Carlos.

F Russell

unread,
Jan 16, 2021, 5:26:31 PM1/16/21
to
On Sat, 16 Jan 2021 08:55:59 -0800, Steve Mysterious wrote:

>
> Is there Linux data recovery software you would recommend?
>

http://foremost.sourceforge.net/



--

Systemd free. D.E. free.

Always and forever.

Bobbie Sellers

unread,
Jan 16, 2021, 6:38:33 PM1/16/21
to
Look for redorescue-3.0.0.iso on www,distrowatch.com
and for rescuezilla-2.1-64bit.groovy.iso

Both are recently promoted live distributions
that might be helpful.

Be sure to check on their websites for use
directions of whatever tools they have
included.
Good luck. Let us know what
happens.

bliss- I have an athletic nose, it runs in all weathers...

--
bliss dash SF 4 ever at dslextreme dot com

Carlos E.R.

unread,
Jan 16, 2021, 7:00:10 PM1/16/21
to
On 16/01/2021 23.25, F Russell wrote:
> On Sat, 16 Jan 2021 08:55:59 -0800, Steve Mysterious wrote:
>
>>
>> Is there Linux data recovery software you would recommend?
>>
>
> http://foremost.sourceforge.net/

I forgot about that one.

On openSUSE there is a security:forensics repository with some extra
tools, including foremost.


--
Cheers, Carlos.

Steve Mysterious

unread,
Jan 16, 2021, 7:17:48 PM1/16/21
to
On Saturday, January 16, 2021 at 3:28:09 PM UTC-5, Carlos E.R. wrote:

> You would have to prepare a Linux system with fstrim disabled, connect
> the failed disk, image it in full to a suitable big hard disk, then
> disconnect the disk. From that moment, you would work on copies of this
> copy.

You are the 3rd person who said the thing to do is to copy an image of the overwritten/formatted drive to another drive.
My computer's main hard drive isn't big enough. The external drive was 1T. I see on Amazon that they are cheap, so I
will get another. Someone recommended dd to make an image. I don't know anything about that, fstrim or these other applications.

I don't need my files on the messed up drive anytime soon. So I will take my time and start learning. I will be saving $624 - $1350 USD - the prices quoted to me by two professional services. Very good motivation.

You will likely see me here again asking questions.

Am I correct that the basic process is to copy an image of the compromised drive to a new drive, then use software to restore file by file over time?

Steve

Steve Mysterious

unread,
Jan 16, 2021, 7:18:50 PM1/16/21
to
Cool. I am on Mint 20, but I'm sure there is a way for me to use those tools.

Thanks

Andrei Z.

unread,
Jan 17, 2021, 2:25:28 AM1/17/21
to
7 of the Best Data Recovery Tools for Linux - Make Tech Easier
https://www.maketecheasier.com/recover-data-linux-tools/

How to Use Foremost to Recover Deleted Files in Linux - Make Tech Easier
https://www.maketecheasier.com/use-foremost-recover-deleted-files-linux/

How to Recover Deleted Files Using TestDisk in Linux
https://www.tecmint.com/recover-deleted-files-using-testdisk-in-linux/

Mint 20.1

:~$ apt show foremost
Package: foremost
Version: 1.5.7-9
...

:~$ apt show testdisk
Package: testdisk
Version: 7.1-5
...
Description: Partition scanner and disk recovery tool, and PhotoRec file
recovery tool
...

Carlos E.R.

unread,
Jan 17, 2021, 4:56:10 AM1/17/21
to
On 17/01/2021 01.17, Steve Mysterious wrote:
> On Saturday, January 16, 2021 at 3:28:09 PM UTC-5, Carlos E.R. wrote:
>
>> You would have to prepare a Linux system with fstrim disabled, connect
>> the failed disk, image it in full to a suitable big hard disk, then
>> disconnect the disk. From that moment, you would work on copies of this
>> copy.
>
> You are the 3rd person who said the thing to do is to copy an image of the overwritten/formatted drive to another drive.
> My computer's main hard drive isn't big enough. The external drive was 1T. I see on Amazon that they are cheap, so I
> will get another. Someone recommended dd to make an image. I don't know anything about that, fstrim or these other applications.
>
> I don't need my files on the messed up drive anytime soon. So I will take my time and start learning. I will be saving $624 - $1350 USD - the prices quoted to me by two professional services. Very good motivation.

You need twice (maybe thrice) the size of the damaged disk at least. One
to make an image, which you do not touch, then another copy in which you
do testing to find out what works. Rotating rust to keep the price low.

>
> You will likely see me here again asking questions.
>
> Am I correct that the basic process is to copy an image of the compromised drive to a new drive, then use software to restore file by file over time?

Yes. If reconstruction of the original filesystem is not possible.

What those tools do is read the disk (sectors marked as deleted but
which weren't actually deleted) to see if the contents seem to be the
start of a known file type, then try find in the disk the next sector of
that file. This is very slow.

And for this to work we need to image the sectors before they are
trimmed and actually erased (some say that an erased sector on rotating
rust may be read in a lab, by reading marginal magnetic fields; maybe
the CIA has that technology, not common labs. On SSD an erased block is
really lost).

I have done this before, but never on SSD. Thus I'm not absolutely sure
about how to impede the trimming.

Trim can be done in two ways: by a command run periodically (cron job,
systemd timer) or fstab option. Ideal would be to find a kernel option
that disables trim, and do the imaging fast.

--
Cheers, Carlos.

The Natural Philosopher

unread,
Jan 17, 2021, 5:08:28 AM1/17/21
to
The problem with an SSD is the layer of remapping inside it.

I don't know if for example a block marked as erased will not in fact be
simply ignored in a read request.

OTOH if its a fairly unused disk, there is a good chance that re
formatting it will simply have created a new directory sector and the
old sectors will still be there unerased, but that's the problem - where
would a block that has been replaced by wear levelling be, and would
the SSD firmware when accessing it later, simply erase it at that time.
Or would it already have erased it?

My guess is, and its only a guess, that a block removed from the active
pool because its previous data has been 'overwritten' would simply be
erased as a background activity, so that it is ready to be written to again.

So directory shit is probably gone.

There shouldn't be any need to erase data sectors though. That may well
exist in its last mapped position as raw sectors. Unless the format
program recognised an SSD and erased it all...


--
“Those who can make you believe absurdities, can make you commit
atrocities.”

― Voltaire, Questions sur les Miracles à M. Claparede, Professeur de
Théologie à Genève, par un Proposant: Ou Extrait de Diverses Lettres de
M. de Voltaire

Carlos E.R.

unread,
Jan 17, 2021, 6:00:09 AM1/17/21
to
On 17/01/2021 11.08, The Natural Philosopher wrote:
> On 17/01/2021 09:54, Carlos E.R. wrote:
>> On 17/01/2021 01.17, Steve Mysterious wrote:
>>> On Saturday, January 16, 2021 at 3:28:09 PM UTC-5, Carlos E.R. wrote:

...

>>
> The problem with an SSD is the layer of remapping inside it.
>
> I don't know if for example a block marked as erased will not in fact be
> simply ignored in a  read request.

It is possible, yes. Depends if the "erased" mark is in the filesystem,
or is inside the SSD.


> OTOH if its a fairly unused disk, there is a good chance that re
> formatting it will simply have created a new directory sector and the
> old sectors will still be there unerased,

That's the hope. That's what happens with rotating rust, the filesystem
knows those sectors are /empty/ and simply overwrites them when there is
nee content (not before), and the filesystem marks them as used. The SSD
for writing needs first to erase, I understand, but it should happen at
that point, not in advance.

> but that's the problem - where
> would a block that has been replaced by wear levelling be,  and would
> the SSD firmware when accessing it later, simply erase it at that time.
> Or would it already have erased it?
>
> My guess is, and its only a guess, that a block removed from the active
> pool because its previous data has been 'overwritten' would simply be
> erased as a background activity, so that it is ready to be written to
> again.
>
> So directory shit is probably gone.
>
> There shouldn't be any need to erase data sectors though. That may well
> exist in its last mapped position as raw sectors. Unless the format
> program recognised an SSD and erased it all...

New territory.

--
Cheers, Carlos.

Steve Mysterious

unread,
Jan 17, 2021, 10:32:08 AM1/17/21
to
I did this in Mint 20 from a menu in Nautilus. Highlight the drive, right click, get a nice little menu.

I learned that path does quick formatting so I have a little bit more hope.

Steve Mysterious

unread,
Jan 17, 2021, 9:13:08 PM1/17/21
to
On Sunday, January 17, 2021 at 4:56:10 AM UTC-5, Carlos E.R. wrote:
> On 17/01/2021 01.17, Steve Mysterious wrote:
> > On Saturday, January 16, 2021 at 3:28:09 PM UTC-5, Carlos E.R. wrote:
> >
> >> You would have to prepare a Linux system with fstrim disabled, connect
> >> the failed disk, image it in full to a suitable big hard disk, then
> >> disconnect the disk. From that moment, you would work on copies of this
> >> copy.
> >
> > You are the 3rd person who said the thing to do is to copy an image of the overwritten/formatted drive to another drive.
> > My computer's main hard drive isn't big enough. The external drive was 1T. I see on Amazon that they are cheap, so I
> > will get another. Someone recommended dd to make an image. I don't know anything about that, fstrim or these other applications.
> >
> > I don't need my files on the messed up drive anytime soon. So I will take my time and start learning. I will be saving $624 - $1350 USD - the prices quoted to me by two professional services. Very good motivation.
> You need twice (maybe thrice) the size of the damaged disk at least. One
> to make an image, which you do not touch, then another copy in which you
> do testing to find out what works. Rotating rust to keep the price low.

I see some nice external SSD 4T drives on Amazon for reasonable prices, especially considering what the recovery shops wanted to charge me. I am going to order one tomorrow morning. Thanks.


chovy

unread,
Jan 18, 2021, 8:39:52 PM1/18/21
to
Honestly, if you're not backing up your data in this day and age...

I use dropbox and git on linux to backup important data.

--
chovy - https://chovy.com

Steve Mysterious

unread,
Jan 19, 2021, 7:25:17 AM1/19/21
to
On Monday, January 18, 2021 at 8:39:52 PM UTC-5, chovy wrote:
> Honestly, if you're not backing up your data in this day and age...
>
> I use dropbox and git on linux to backup important data.

My strong preferences is not to have some corporation have my family photos, old tax documents, medical records, diary entries etc on their servers.

The new 4TB external hard drive I ordered to fix the 1T external hard drive will be a great place for a backup.

chovy

unread,
Jan 19, 2021, 10:35:55 PM1/19/21
to
That's why you encrypt your data first with gpg.

Computer Nerd Kev

unread,
Jan 21, 2021, 4:49:21 PM1/21/21
to
Technically that's no guarantee that it won't be trivially
decryptable in 20 years time with whatever thousand-core super
CPU, or miniature quantum computing unit, that everyone might need
just to run the latest Windows 17, is able to brute-force that
encryption without breaking sweat. Just like how common encryption
technologies from the 90s are able to be cracked today.

--
__ __
#_ < |\| |< _#

Steve Mysterious

unread,
Jan 24, 2021, 8:46:21 PM1/24/21
to
On Friday, January 15, 2021 at 10:32:25 PM UTC-5, Steve Mysterious wrote:
Hi all,

I bought a 4T external SSD hard drive to copy the 1T drive to.

I ran this command, which took about 3+ hours to complete

sudo dd if=/dev/sdc1 of=/dev/sdb2 bs=64K conv=noerror,sync

I had a look at the 4T drive via nautilus and it is indicating nothing was copied there.

Am I missing something, or is everything from my compromised 1T drive completely gone and dd just jerked around for 3 hours?

Did I use the wrong command to make an image of the 1T drive and put it on the 4T drive?

Thanks

Steve

Tauno Voipio

unread,
Jan 25, 2021, 3:53:51 AM1/25/21
to
You attempted to make a partition copy from partition 1 of /dev/sdc
to partition 2 of /dev/sdb. At least, the target SSD should have a
compatible partition to copy to. I guess thst you did not create it.

For such a large copy, the block size should rather be 1M.

--

-TV

Steve Mysterious

unread,
Jan 25, 2021, 8:25:18 AM1/25/21
to
Thanks for the reply.

I'm new to dd

What SHOULD have been the command to make an image of the 1T drive and to copy it to the 4T drive?

I'm trying to rescue overwritten data from the 1T drive.

Thanks for the response

Steve

Carlos E.R.

unread,
Jan 25, 2021, 9:00:10 AM1/25/21
to
On 25/01/2021 14.25, Steve Mysterious wrote:
> On Monday, January 25, 2021 at 3:53:51 AM UTC-5, Tauno Voipio wrote:
>> On 25.1.21 3.46, Steve Mysterious wrote:
>>> On Friday, January 15, 2021 at 10:32:25 PM UTC-5, Steve Mysterious wrote:

...

>>> Hi all,
>>>
>>> I bought a 4T external SSD hard drive to copy the 1T drive to.
>>>
>>> I ran this command, which took about 3+ hours to complete
>>>
>>> sudo dd if=/dev/sdc1 of=/dev/sdb2 bs=64K conv=noerror,sync
>>>
>>> I had a look at the 4T drive via nautilus and it is indicating nothing was copied there.
>>>
>>> Am I missing something, or is everything from my compromised 1T drive completely gone and dd just jerked around for 3 hours?
>>>
>>> Did I use the wrong command to make an image of the 1T drive and put it on the 4T drive?
>>>
>>> Thanks
>>>
>>> Steve
>> You attempted to make a partition copy from partition 1 of /dev/sdc
>> to partition 2 of /dev/sdb. At least, the target SSD should have a
>> compatible partition to copy to. I guess thst you did not create it.
>>
>> For such a large copy, the block size should rather be 1M.
>>

>
> Thanks for the reply.
>
> I'm new to dd
>
> What SHOULD have been the command to make an image of the 1T drive and to copy it to the 4T drive?
>
> I'm trying to rescue overwritten data from the 1T drive.

To rescue the *drive*:

dd if=/dev/sdc of=/dev/sdb bs=1M conv=noerror,sync



--
Cheers, Carlos.

The Natural Philosopher

unread,
Jan 25, 2021, 9:28:01 AM1/25/21
to
and it is unlikely that the disk will in fact show any data once you do
it as far as a *file browser* is concerned, because it was formatted
wasn't it?


--
Climate Change: Socialism wearing a lab coat.

Tauno Voipio

unread,
Jan 25, 2021, 10:34:11 AM1/25/21
to
A raw copy contains the filesystem data structures, otherwise created
by mkfs (which is often mis-called formatting), so the target drive
does not have to have a filesystem on it. If the target drive is a
soft-sectored oldish spinning-rust device, it may need formatting,
see note below.

A raw copy of a full drive contains also the partitioning of the
original drive, so the 4T drive will mount as a 1T drive.

Please note that in either case, the image has to be mounted to show
its contents as files.

Note:

Formatting appies to soft-sectored spinning-rust disks. Its
purpose is to paint the sector boundaries on the disk surface.
The misuse of the term comes maybe from MS-DOS, where the same
utility both formatted and created the FAT filesystem.

--

-TV

Carlos E.R.

unread,
Jan 25, 2021, 10:44:11 AM1/25/21
to
And wait, this is not what I recommended, because this way you obtain a
single disk of 1TB wasting those 4 TB.


What I recommended way back, was:

* Partition and format the destination disk however you wish, with a
linux filesystem. Notice that any of the previous dd operations
destroyed whatever was stored on that disk previously.

* Mount it somewhere, say /mnt/bigdata.
* Image the bad disk to a *file*. Assuming it is /dev/sdc, then you do:

dd if=/dev/sdc of=OriginalBadDisk.img bs=1M conv=noerror,sync

* disconnect the bad disk.
* work on the file "OriginalBadDisk.img".


--
Cheers, Carlos.

Steve Mysterious

unread,
Mar 7, 2021, 12:18:38 PM3/7/21
to
On Friday, January 15, 2021 at 10:32:25 PM UTC-5, Steve Mysterious wrote:
> I have an external SSD hard drive, about 1 Terabyte.
>
> The other day I was in a rush, I was trying to format a thumb drive, and accidentally formatted the SSD hard drive instead.
>
> I read that it is harder to recover data from an SSD hard drive due to the different technology.
>
> I heard about TestDisk and PhotoRec.
>
> If I try those two applications before going to a data recovery service is it possible to the situation worse in terms of recovering data?
>
> Thanks for any opinions.
>
> Steve

I thought I would report back with the end of the story given that so many people on comp.os.linux.misc offered solid advice.

I recieved dour predictions of recovering data from an external SSD USB drive.

Microcenter wanted $200 to tell me if the data could be recovered, $1350 - $3,000+ to do the job. A local computer shop $675. I found out that the formatting that the Mint file manager used was quick formatting. So despite being a scared newbie, I decided to take a few weekends to give it go by asking questions and trying things out.

This is what I did:

1. I immediately disconnected the 1T external USB SSD drive where the data was accidentally erased.
I kept it in a drawer, safe from any automatic processes that might have destroyed more data.

2. I bought a 4T external USB SSD drive to do my data restoration attempts on.

3. I plugged both drives into my computer and ran [b]lsblk [/b]to see what their locations were

1T drive was at /dev/sdc
4T drive was at /dev/sdb

4. I partitioned the 4T drive to have a massive area for multiple drive images

/dev/sdb2 /


5. I then mounted that big partition on the 4T drive as "BigDrive"

sudo mount /dev/sdb2 /home/steve/BigDrive


6. I then made an image of the 1T drive, outputting it to the big partition on the 4T drive Took about 5-7 hours.

dd if=/dev/sdc of=/home/steve/BigDrive/OriginalBadDisk.img bs=1M conv=noerror,sync status=progress


7. I then played it safe by making a copy of that image for each different rescue attempt I made. It took about 5 hours per copy

8. I then launched TestDisk so it would read my drive image instead of reading a physical drive ( the default ):

$ cd /home/steve/BigDrive/
$ mkdir Rescued
$ testdisk OriginalBadDiskCopy2.img

The "Rescued" directory was to store what TestDisk rescued.

TestDisk, though being a command line interface application has a menu system similar to ncurses/old DOS programs.

TestDisk found the original "partition" in OriginalBadDiskCopy2.img

From that point I just navigated through my "lost" directories selecting which files and which subdirectories (TestDisk had the original file and directory names ) to copy over to my directory "Rescued".

TestDisk only failed to copy over 1 video file and 1 picture file out of a massive multimedia collection.

There were a few dozen rescue failures for files of other types, but most were things I did not care about or that I could replace.

I used this [b]short[/b] TestDisk tutorial to get going:

[b]How to Recover Deleted Files Using TestDisk in Linux[/b]
https://www.tecmint.com/recover-deleted-files-using-testdisk-in-linux/

My first rescue attempt was with the software called Foremost.
It gave me all of the rescued data back sorted into subdirectories by file extension.
None of the files had names, just numbers Foremost assigned to the files in lieu of the original names.
I only got 2 usable jpeg files and 1 usable mp4 out of the 7 hour attempt.
I could not launch the rest of the files it recovered.
Even if I could, the files would have had no names and were scrambled out of order.
TestDisk was definitely the way to go.


Carlos E.R.

unread,
Mar 7, 2021, 4:40:09 PM3/7/21
to
On 07/03/2021 18.18, Steve Mysterious wrote:
> On Friday, January 15, 2021 at 10:32:25 PM UTC-5, Steve Mysterious wrote:
>> I have an external SSD hard drive, about 1 Terabyte.
>>
>> The other day I was in a rush, I was trying to format a thumb drive, and accidentally formatted the SSD hard drive instead.
>>
>> I read that it is harder to recover data from an SSD hard drive due to the different technology.
>>
>> I heard about TestDisk and PhotoRec.
>>
>> If I try those two applications before going to a data recovery service is it possible to the situation worse in terms of recovering data?
>>
>> Thanks for any opinions.
>>
>> Steve
>
> I thought I would report back with the end of the story given that so many people on comp.os.linux.misc offered solid advice.
>
> I recieved dour predictions of recovering data from an external SSD USB drive.
>
> Microcenter wanted $200 to tell me if the data could be recovered, $1350 - $3,000+ to do the job. A local computer shop $675. I found out that the formatting that the Mint file manager used was quick formatting. So despite being a scared newbie, I decided to take a few weekends to give it go by asking questions and trying things out.

Aha.

>
> This is what I did:
>
> 1. I immediately disconnected the 1T external USB SSD drive where the data was accidentally erased.
> I kept it in a drawer, safe from any automatic processes that might have destroyed more data.
>
> 2. I bought a 4T external USB SSD drive to do my data restoration attempts on.
>
> 3. I plugged both drives into my computer and ran [b]lsblk [/b]to see what their locations were
>
> 1T drive was at /dev/sdc
> 4T drive was at /dev/sdb
>
> 4. I partitioned the 4T drive to have a massive area for multiple drive images
>
> /dev/sdb2 /

Perfect :-)



> 5. I then mounted that big partition on the 4T drive as "BigDrive"
>
> sudo mount /dev/sdb2 /home/steve/BigDrive
>
>
> 6. I then made an image of the 1T drive, outputting it to the big partition on the 4T drive Took about 5-7 hours.
>
> dd if=/dev/sdc of=/home/steve/BigDrive/OriginalBadDisk.img bs=1M conv=noerror,sync status=progress

Very good.



> 7. I then played it safe by making a copy of that image for each different rescue attempt I made. It took about 5 hours per copy

Yes.

>
> 8. I then launched TestDisk so it would read my drive image instead of reading a physical drive ( the default ):
>
> $ cd /home/steve/BigDrive/
> $ mkdir Rescued
> $ testdisk OriginalBadDiskCopy2.img

Right.


> The "Rescued" directory was to store what TestDisk rescued.
>
> TestDisk, though being a command line interface application has a menu system similar to ncurses/old DOS programs.
>
> TestDisk found the original "partition" in OriginalBadDiskCopy2.img

How long did it take?


> From that point I just navigated through my "lost" directories selecting which files and which subdirectories (TestDisk had the original file and directory names ) to copy over to my directory "Rescued".
>
> TestDisk only failed to copy over 1 video file and 1 picture file out of a massive multimedia collection.
>
> There were a few dozen rescue failures for files of other types, but most were things I did not care about or that I could replace.

Marvelous :-)

>
> I used this [b]short[/b] TestDisk tutorial to get going:
>
> [b]How to Recover Deleted Files Using TestDisk in Linux[/b]
> https://www.tecmint.com/recover-deleted-files-using-testdisk-in-linux/
>
> My first rescue attempt was with the software called Foremost.

A File carver. Very slow but good. File carving is the last resource.
First testdisk type, to try reconstruct the partition table/filesystem.

> It gave me all of the rescued data back sorted into subdirectories by file extension.
> None of the files had names, just numbers Foremost assigned to the files in lieu of the original names.
> I only got 2 usable jpeg files and 1 usable mp4 out of the 7 hour attempt.
> I could not launch the rest of the files it recovered.
> Even if I could, the files would have had no names and were scrambled out of order.
> TestDisk was definitely the way to go.

Yep.

Thank you for telling us! :-)

Ok, so it worked the same way as with rotating rust. That was the part I
did not know.


--
Cheers, Carlos.

Steve Mysterious

unread,
Mar 7, 2021, 6:31:02 PM3/7/21
to
On Sunday, March 7, 2021 at 4:40:09 PM UTC-5, Carlos E.R. wrote:

> > TestDisk found the original "partition" in OriginalBadDiskCopy2.img
> How long did it take?

About 5-6 hours. It went through a "cylinder" by cylinder analysis on the drive image (1T), thankfully with a percentage-done readout.
All tools should have that standard instead of a "Please wait". I ran into a tool along my journey that probably would have taken all night and would have given no indication it would was doing anything.


> Thank you for telling us! :-)
>
> Ok, so it worked the same way as with rotating rust. That was the part I
> did not know.

I think the saving graces in my situation were that the nautilus file manager in Mint & Cinnamon only did quick formatting and that my drive was completely undamaged.

Steve

Antti Talsta

unread,
Mar 8, 2021, 1:04:34 AM3/8/21
to
On 2021-03-07, Steve Mysterious <tink...@gmail.com> wrote:
> TestDisk was definitely the way to go.

Nice to read a success story for a change.

--
Antti Talsta

Carlos E.R.

unread,
Mar 8, 2021, 8:44:11 AM3/8/21
to
On 08/03/2021 00.30, Steve Mysterious wrote:
> On Sunday, March 7, 2021 at 4:40:09 PM UTC-5, Carlos E.R. wrote:
>
>>> TestDisk found the original "partition" in OriginalBadDiskCopy2.img
>> How long did it take?
>
> About 5-6 hours. It went through a "cylinder" by cylinder analysis on the drive image (1T), thankfully with a percentage-done readout.
> All tools should have that standard instead of a "Please wait". I ran into a tool along my journey that probably would have taken all night and would have given no indication it would was doing anything.

The typical default in Linux is say nothing unless there is an error.
Sometimes there is an option to show progress.

>
>
>> Thank you for telling us! :-)
>>
>> Ok, so it worked the same way as with rotating rust. That was the part I
>> did not know.
>
> I think the saving graces in my situation were that the nautilus file manager in Mint & Cinnamon only did quick formatting and that my drive was completely undamaged.

This is the default for all formatting/partitioning tools (in Linux at
least). Only the metadata area is reset (not necessarily erased).

Reasoning is, there is no longer the need to low level format hard disk
(even impossible), and erasing all the sectors, in rotating disk takes
hours, and in SSD disks reduces the life of the disk significantly.


--
Cheers, Carlos.

Pascal Hambourg

unread,
Mar 8, 2021, 9:40:47 AM3/8/21
to
Le 08/03/2021 à 14:43, Carlos E.R. a écrit :
> On 08/03/2021 00.30, Steve Mysterious wrote:
>>
>> I think the saving graces in my situation were that the nautilus file
>> manager in Mint & Cinnamon only did quick formatting and that my drive
>> was completely undamaged.
>
> This is the default for all formatting/partitioning tools (in Linux at
> least). Only the metadata area is reset (not necessarily erased).

Not all tools. The default for mkntfs from ntfs-3g is a "long" format.

> Reasoning is, there is no longer the need to low level format hard disk
> (even impossible), and erasing all the sectors, in rotating disk takes
> hours, and in SSD disks reduces the life of the disk significantly.

But discard (TRIM) of all free blocks is fast and harmless.
Guess what ? it seems to be the default for mke2fs, mkfs.btrfs and
mkfs.xfs. Beware !

Pascal Hambourg

unread,
Mar 8, 2021, 10:14:57 AM3/8/21
to
Le 16/01/2021 à 21:26, Carlos E.R. a écrit :
>
> If you only formatted the disk there is a good chance of recovering most
> of the data.
>
> You would have to prepare a Linux system with fstrim disabled

Or with automatic mounting disabled. fstrim works only on mounted
filesystems.

Pascal Hambourg

unread,
Mar 8, 2021, 10:19:26 AM3/8/21
to
Le 17/01/2021 à 11:59, Carlos E.R. a écrit :
>
> The SSD
> for writing needs first to erase, I understand, but it should happen at
> that point, not in advance.

On the contrary, erasure should happen in advance before writing. This
is the whole point of TRIM and garbage collection. Erasure is slow and
would degrade write speed if it was done just before writing.

Aragorn

unread,
Mar 8, 2021, 5:09:23 PM3/8/21
to
On 08.03.2021 at 16:14, Pascal Hambourg scribbled:
And for that matter, only if they are mounted read/write.

--
With respect,
= Aragorn =

Carlos E.R.

unread,
Mar 8, 2021, 9:20:09 PM3/8/21
to
On 08/03/2021 15.40, Pascal Hambourg wrote:
> Le 08/03/2021 à 14:43, Carlos E.R. a écrit :
>> On 08/03/2021 00.30, Steve Mysterious wrote:
>>>
>>> I think the saving graces in my situation were that the nautilus file
>>> manager in Mint & Cinnamon only did quick formatting and that my
>>> drive was completely undamaged.
>>
>> This is the default for all formatting/partitioning tools (in Linux at
>> least). Only the metadata area is reset (not necessarily erased).
>
> Not all tools. The default for mkntfs from ntfs-3g is a "long" format.

Why?

Something in the ntfs filesystem assumes that a sector is zeroed?

>
>> Reasoning is, there is no longer the need to low level format hard
>> disk (even impossible), and erasing all the sectors, in rotating disk
>> takes hours, and in SSD disks reduces the life of the disk significantly.
>
> But discard (TRIM) of all free blocks is fast and harmless.
> Guess what ? it seems to be the default for mke2fs, mkfs.btrfs and
> mkfs.xfs. Beware !


--
Cheers, Carlos.

Keith Scarlett

unread,
Sep 14, 2021, 10:40:49 AM9/14/21
to
Hello,

Firstly thanks all for the contributions, really useful. I have done something very similar (identical?) i.e. I was trying to format (write a disk image) to a memory stick while I also had a 500GB external drive attached and unfortunately started to write the image to the 500GB drive. I think I noticed after approx. 30 seconds.

What value is there in me trying to use TestDisk as suggested in this thread? Before I use TestDisk is it worth trying to PhotoRec first? The data on the 500GB is important so any help / guidance much appreciated. I have a Linux Mint system and I was trying to write the image using 'USB Stick Writer', a software that's included with Linux Mint installations.

Thanks,

Keith

Carlos E. R.

unread,
Sep 14, 2021, 11:22:37 AM9/14/21
to
On 14/09/2021 16.40, Keith Scarlett wrote:

...

>
> Hello,
>
> Firstly thanks all for the contributions, really useful. I have done something very similar (identical?) i.e. I was trying to format (write a disk image) to a memory stick while I also had a 500GB external drive attached and unfortunately started to write the image to the 500GB drive. I think I noticed after approx. 30 seconds.
>
> What value is there in me trying to use TestDisk as suggested in this thread? Before I use TestDisk is it worth trying to PhotoRec first? The data on the 500GB is important so any help / guidance much appreciated. I have a Linux Mint system and I was trying to write the image using 'USB Stick Writer', a software that's included with Linux Mint installations.

How big was that memory stick?

The first step always should be to clone the disk to another one or as
image file. Then you work at the image.

If at all possible, you would clone it first, mark it read only, then a
copy of the copy and work on that one. If you have to restart, you
create another copy from the initial first copy, not the original.

The first attempt should be testdisk.

If that fails, recreate the copy and attempt photorec or similar tool.

Photorec scans all the sectors looking for known types of files, mostly
photos. It is possible that other tools scan better other types of
files. It is a very slow procedure.

You should also consider "foremost".

Any other attempt, create a secondary copy and work on it, never on the
original.

--
Cheers,
Carlos E.R.

Keith Scarlett

unread,
Sep 16, 2021, 3:21:07 PM9/16/21
to
> How big was that memory stick?
>
> The first step always should be to clone the disk to another one or as
> image file. Then you work at the image.
>
> If at all possible, you would clone it first, mark it read only, then a
> copy of the copy and work on that one. If you have to restart, you
> create another copy from the initial first copy, not the original.
>
> The first attempt should be testdisk.
>
> If that fails, recreate the copy and attempt photorec or similar tool.
>
> Photorec scans all the sectors looking for known types of files, mostly
> photos. It is possible that other tools scan better other types of
> files. It is a very slow procedure.
>
> You should also consider "foremost".
>
> Any other attempt, create a secondary copy and work on it, never on the
> original.
>
> --
> Cheers,
> Carlos E.R.

Thank-you for the reply. So the memory stick was 15GB (the intended write target) but the external HDD I accidentally wrote the disk image to was 500GB.

Using the dd command should I be trying to clone the drive or write an image of that drive? Are these actually different procedures or am I confused?

Based on the OP's experience it seems I'll have to get a 2TB external drive (then partition this) if I want to clone the 500GB external drive and then have a copy of that, does that sound right?

In TestDisk how obvious is it that it has found the correct / 'deleted' partition, what should I be looking for after TestDisk has analysed the cloned image? I'm 99.99% sure it would have been a FAT partition about 500GB in size and that anything created during the accidental write would be something else, perhaps ISO?

What is the likelihood of full data recovery given the circumstances? What is the likelihood of partial data recovery?

Thanks,

Keith

Carlos E. R.

unread,
Sep 16, 2021, 8:55:08 PM9/16/21
to
On 16/09/2021 21.21, Keith Scarlett wrote:
>> How big was that memory stick?
>>
>> The first step always should be to clone the disk to another one or as
>> image file. Then you work at the image.
>>
>> If at all possible, you would clone it first, mark it read only, then a
>> copy of the copy and work on that one. If you have to restart, you
>> create another copy from the initial first copy, not the original.
>>
>> The first attempt should be testdisk.
>>
>> If that fails, recreate the copy and attempt photorec or similar tool.
>>
>> Photorec scans all the sectors looking for known types of files, mostly
>> photos. It is possible that other tools scan better other types of
>> files. It is a very slow procedure.
>>
>> You should also consider "foremost".
>>
>> Any other attempt, create a secondary copy and work on it, never on the
>> original.


> Thank-you for the reply. So the memory stick was 15GB (the intended write target) but the external HDD I accidentally wrote the disk image to was 500GB.
>
> Using the dd command should I be trying to clone the drive or write an image of that drive? Are these actually different procedures or am I confused?

Depends on what you have available to write the copy to. For example, if
you have a 4TB disk, create an image copy.

Example.

Corrupted disk is /dev/sdX

Empty 4 TB disk is /dev/sdY, and you mount it on /workspace

You would do:

dd if=/dev/sdX of=/workspace/ClonedImageOfCorruptedDisk.img bs=100MB


(4TB happens to be the sweet spot of price/size where I live)

> Based on the OP's experience it seems I'll have to get a 2TB external drive (then partition this) if I want to clone the 500GB external drive and then have a copy of that, does that sound right?

That would be nice, yes.


> In TestDisk how obvious is it that it has found the correct / 'deleted' partition, what should I be looking for after TestDisk has analysed the cloned image? I'm 99.99% sure it would have been a FAT partition about 500GB in size and that anything created during the accidental write would be something else, perhaps ISO?

Ideally, you could mount the 500GB repaired image and you could read it
- except the first 15GB, so the original directory structure would be
lost. Hopefully there would be a "reconstructed" structure of invented
names.


>
> What is the likelihood of full data recovery given the circumstances? What is the likelihood of partial data recovery?

Partial data recovery, very good. What software will finally manage to
do it, I don't know. You have to try and select the best result...


Look, I know better how to actually do it (following my nose) than
describe it.


--
Cheers,
Carlos E.R.

Pascal Hambourg

unread,
Sep 17, 2021, 1:30:07 AM9/17/21
to
Le 17/09/2021 à 02:53, Carlos E. R. a écrit :
> On 16/09/2021 21.21, Keith Scarlett wrote:
>
>> So the memory stick was 15GB (the intended write target) but the external HDD I accidentally wrote the disk image to was 500GB.
(...)
> Ideally, you could mount the 500GB repaired image and you could read it
> - except the first 15GB

Why 15 GB ? The intended destination drive size is irrelevant. What
matters is the written image size or the write speed (45 MB/s max if USB
2, more if USB 3) * write time, whichever is higher.

Carlos E. R.

unread,
Sep 17, 2021, 6:17:15 AM9/17/21
to
Maybe you missed that he overwrote his good disk with an image for an
USB stick of 15 GB. It is not the destination drive, it is the max
corruption size.

Knowing the actual size of the source image would be better. If it was a
partial write, dd would say how much it actually wrote.

--
Cheers,
Carlos E.R.

The Natural Philosopher

unread,
Sep 17, 2021, 8:24:06 AM9/17/21
to
Problem is that the first thing to get written would be the directory tracks

there are other block metadata areas in ext2/4 but thats the main one

Data may still be there, but no reason to suppose its more than a random
collection of sectors.

--
When plunder becomes a way of life for a group of men in a society, over
the course of time they create for themselves a legal system that
authorizes it and a moral code that glorifies it.

Frédéric Bastiat

Carlos E. R.

unread,
Sep 17, 2021, 8:59:15 AM9/17/21
to
On 17/09/2021 14.24, The Natural Philosopher wrote:
> On 17/09/2021 11:15, Carlos E. R. wrote:
>> On 17/09/2021 07.30, Pascal Hambourg wrote:
>>> Le 17/09/2021 à 02:53, Carlos E. R. a écrit :
>>>> On 16/09/2021 21.21, Keith Scarlett wrote:
>>>>
>>>>> So the memory stick was 15GB (the intended write target) but the
>>>>> external HDD I accidentally wrote the disk image to was 500GB.
>>> (...)
>>>> Ideally, you could mount the 500GB repaired image and you could read it
>>>> - except the first 15GB
>>>
>>> Why 15 GB ? The intended destination drive size is irrelevant. What
>>> matters is the written image size or the write speed (45 MB/s max if USB
>>> 2, more if USB 3) * write time, whichever is higher.
>>
>> Maybe you missed that he overwrote his good disk with an image for an
>> USB stick of 15 GB. It is not the destination drive, it is the max
>> corruption size.
>>
>> Knowing the actual size of the source image would be better. If it was a
>> partial write, dd would say how much it actually wrote.
>>
> Problem is that the first thing to get written would be the directory
> tracks

Yep.

>
> there are other block metadata areas in ext2/4 but thats the main one

I think he said it was FAT. If that is so, the FAT table would be gone,
and then the master directory structure. Possibly some subdirectories
tables would remain.

>
> Data may still be there, but no reason to suppose its more than a random
> collection of sectors.

Even in that case, file carving tools manage good results. Not the file
names in this case, though.


--
Cheers,
Carlos E.R.

Pascal Hambourg

unread,
Sep 17, 2021, 5:47:02 PM9/17/21
to
Le 17/09/2021 à 12:15, Carlos E. R. a écrit :
> On 17/09/2021 07.30, Pascal Hambourg wrote:
>> Le 17/09/2021 à 02:53, Carlos E. R. a écrit :
>>>
>>> Ideally, you could mount the 500GB repaired image and you could read it
>>> - except the first 15GB
>>
>> Why 15 GB ? The intended destination drive size is irrelevant. What
>> matters is the written image size or the write speed (45 MB/s max if USB
>> 2, more if USB 3) * write time, whichever is higher.
>
> Maybe you missed that he overwrote his good disk with an image for an
> USB stick of 15 GB.

Again, the USB stick size is irrelevant.
Where did you see that the image size was 15 GB ?

Carlos E. R.

unread,
Sep 17, 2021, 8:51:31 PM9/17/21
to
It is obvious, no?

--
Cheers,
Carlos E.R.

Pascal Hambourg

unread,
Sep 18, 2021, 3:30:48 AM9/18/21
to
No. I, as most people, usually write images on USB sticks bigger than
the image size.

Carlos E. R.

unread,
Sep 18, 2021, 5:33:36 AM9/18/21
to
Right, but it would not be bigger than 15GB ;-)

--
Cheers,
Carlos E.R.
0 new messages