Recover deleted JBOD Raid

[Axtux -]

Hello,

I deleted my JBOD raid over 2 HDD but I didn't do anything else. Is there anyway I can recover my filesystem ? I connected my drives to an ubuntu computer with mdadm to try recovering but I'm afraid to make something that would erase data forever. I know I can recover some files with TestDisk but if I can get folders and files structure, I would prefer.

Whole story : I own 2 DNS-320L and I had already installed Alt-F on one of them. I was very happy with it so I decided to install Alt-F on the other one. There was a JBOD raid with 2x 4To HDD. Alt-F mounted this raid read-only. I tried to change mount option without luck. I created a new JBOD raid from my 2 HDD but that didn't solve my trouble so I deleted it. When I deleted this newly created raid, the existing JBOD raid was deleted too. Now I can't access any data anymore.

Thanks in advance for any help,
Axtux

[João Cardoso]

On Monday, 12 October 2015 20:47:14 UTC+1, Axtux - wrote:

Hello,

I deleted my JBOD raid over 2 HDD but I didn't do anything else. Is there anyway I can recover my filesystem ? I connected my drives to an ubuntu computer with mdadm to try recovering but I'm afraid to make something that would erase data forever. I know I can recover some files with TestDisk but if I can get folders and files structure, I would prefer.

Whole story : I own 2 DNS-320L and I had already installed Alt-F on one of them. I was very happy with it so I decided to install Alt-F on the other one. There was a JBOD raid with 2x 4To HDD. Alt-F mounted this raid read-only. I tried to change mount option without luck.

No, you should 'fsck' the filesystem. Alt-F always start ' fsck' at start in automatic mode, and if it does not succeeds the filesystem is mounted read-only and the Status page should show the fsck diagnostic message, generally  a "run me manually". That can be accomplished under Disk->Filesystem->FS Operations, Force Fix, on the right filesystem line.

 

I created a new JBOD raid from my 2 HDD but that didn't solve my trouble so I deleted it. When I deleted this newly created raid, the existing JBOD raid was deleted too. Now I can't access any data anymore.

How did you create and deleted the "new" JBOD? Using the webUI or the command line? What device components? Knowing exactly how and in what sequence you did could help.

When you first created the JBOD, the original RAID *information* should have been erased, but only that, not the data itself. This depends on which device you said the JBOD should be created -- the same components as the original read-only one? most probably, sda2/sdb2?

If you have not created a filesystem on the newly created JBOD RAID, and the RAID components was the same as in the original JBOD, your data should be safe. You only have to create the JBOD again using the same device components and, for safety, the same RAID metadata. Using the webUI will use a newer RAID metadata than the original.

Also, JBOD creation or assembling *might* depends on the device names (sda/sdb), and this depends on whether you swapped the drives or not (left/right bay).

Please post the output of the following commands, which only diagnose the current situation:

cat /proc/mdstat
mdadm --examine --scan --verbose

[Axtux -]

On Tuesday, October 13, 2015 at 7:56:30 PM UTC+2, João Cardoso wrote:

On Monday, 12 October 2015 20:47:14 UTC+1, Axtux - wrote:

Hello,

I deleted my JBOD raid over 2 HDD but I didn't do anything else. Is there anyway I can recover my filesystem ? I connected my drives to an ubuntu computer with mdadm to try recovering but I'm afraid to make something that would erase data forever. I know I can recover some files with TestDisk but if I can get folders and files structure, I would prefer.

Whole story : I own 2 DNS-320L and I had already installed Alt-F on one of them. I was very happy with it so I decided to install Alt-F on the other one. There was a JBOD raid with 2x 4To HDD. Alt-F mounted this raid read-only. I tried to change mount option without luck.

No, you should 'fsck' the filesystem. Alt-F always start ' fsck' at start in automatic mode, and if it does not succeeds the filesystem is mounted read-only and the Status page should show the fsck diagnostic message, generally  a "run me manually". That can be accomplished under Disk->Filesystem->FS Operations, Force Fix, on the right filesystem line.

Yes, I get this message :

Unable to automatically fix sdb2, mounting Read Only: fsck 1.41.14 (22-Dec-2010)
/dev/sdb2: The filesystem size (according to the superblock) is 1952197472 blocks
The physical size of the device is 976098765 blocks
Either the superblock or the partition table is likely to be corrupt!

 The thing is my data was a JBOD on sda2 and sdb2, and I don't see sda2 any more.

 

 

I created a new JBOD raid from my 2 HDD but that didn't solve my trouble so I deleted it. When I deleted this newly created raid, the existing JBOD raid was deleted too. Now I can't access any data anymore.

How did you create and deleted the "new" JBOD? Using the webUI or the command line? What device components? Knowing exactly how and in what sequence you did could help.

Yes, already existing RO JBOD RAID was on md1 and I created the new RAID on md2 using webUI. I used exactly same components as original RAID. Then, I deleted this md2 raid using webUI.
 

When you first created the JBOD, the original RAID *information* should have been erased, but only that, not the data itself. This depends on which device you said the JBOD should be created -- the same components as the original read-only one? most probably, sda2/sdb2?

 Exactly same components as original RAID. sda2 and sdb2.

If you have not created a filesystem on the newly created JBOD RAID, and the RAID components was the same as in the original JBOD, your data should be safe. You only have to create the JBOD again using the same device components and, for safety, the same RAID metadata. Using the webUI will use a newer RAID metadata than the original.

No, didn't create anything above the old RAID. I assume I should not use webUI to create it again.

Also, JBOD creation or assembling *might* depends on the device names (sda/sdb), and this depends on whether you swapped the drives or not (left/right bay).

I took a picture of drives (with serial number) before removing them from NAS so I'm able to put them back as they were.

Please post the output of the following commands, which only diagnose the current situation:

cat /proc/mdstat
mdadm --examine --scan --verbose

$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sdb1[0] sda1[1]
      524224 blocks [2/2] [UU]
     
unused devices: <none>
$ mdadm --examine --scan --verbose
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=320150c3:466b1626:b11ebfcf:8a0c7316
   devices=/dev/sdb1,/dev/sda1

This seems to be a RAID1 created by original FW to store some data but nothing about my JBOD.
 

 

Thanks in advance for any help,
Axtux

Regarding those info, do you have any clue ? I'm still new to Linux so I don't know that much and I'm very afraid to do anything wrong.

EDIT : existing filesystems http://postimg.org/image/465111oil/
webUI let me select only sda1 and sdb1 see http://postimg.org/image/vn8mpspy5/

[João Cardoso]

On Wednesday, 14 October 2015 02:16:20 UTC+1, Axtux - wrote:

On Tuesday, October 13, 2015 at 7:56:30 PM UTC+2, João Cardoso wrote:

On Monday, 12 October 2015 20:47:14 UTC+1, Axtux - wrote:

Hello,

I deleted my JBOD raid over 2 HDD but I didn't do anything else. Is there anyway I can recover my filesystem ? I connected my drives to an ubuntu computer with mdadm to try recovering but I'm afraid to make something that would erase data forever. I know I can recover some files with TestDisk but if I can get folders and files structure, I would prefer.

Whole story : I own 2 DNS-320L and I had already installed Alt-F on one of them. I was very happy with it so I decided to install Alt-F on the other one. There was a JBOD raid with 2x 4To HDD. Alt-F mounted this raid read-only. I tried to change mount option without luck.

No, you should 'fsck' the filesystem. Alt-F always start ' fsck' at start in automatic mode, and if it does not succeeds the filesystem is mounted read-only and the Status page should show the fsck diagnostic message, generally  a "run me manually". That can be accomplished under Disk->Filesystem->FS Operations, Force Fix, on the right filesystem line.

Yes, I get this message :

Unable to automatically fix sdb2,

ah, sdb2, not md1 or mtd2! for some reason sdb2 contains an ext filesystem signature and Alt-F tried to mount it. sdb2 should have a RAID signature, and only the assembled sda2+sdb2 RAID device md1 (or md2) should have an ext filesystem signature.

 

 mounting Read Only: fsck 1.41.14 (22-Dec-2010)
/dev/sdb2: The filesystem size (according to the superblock) is 1952197472 blocks
The physical size of the device is 976098765 blocks

The ext filesystem that starts on sdb2 (and continued on sda2) has 1952 Mega blocks, but sdb2 itself only has 976 Mega blocks. Remember,

sda2+sdb2 combined together on md1, (or md2), the JBOD RAID where the filesystem was created on. Read the online RAID and Filesystem help (the (?) icon near the page title.

 

Either the superblock or the partition table is likely to be corrupt!

 The thing is my data was a JBOD on sda2 and sdb2, and I don't see sda2 any more.

Clarify "I don't see sda2 any more". Where don't you see it?

 

 

I created a new JBOD raid from my 2 HDD but that didn't solve my trouble so I deleted it. When I deleted this newly created raid, the existing JBOD raid was deleted too. Now I can't access any data anymore.

How did you create and deleted the "new" JBOD? Using the webUI or the command line? What device components? Knowing exactly how and in what sequence you did could help.

Yes, already existing RO JBOD RAID was on md1 and I created the new RAID on md2 using webUI. I used exactly same components as original RAID. Then, I deleted this md2 raid using webUI.

This should be a webgui bug deployed by the way D-Link created fs and RAID...

 

 

When you first created the JBOD, the original RAID *information* should have been erased, but only that, not the data itself. This depends on which device you said the JBOD should be created -- the same components as the original read-only one? most probably, sda2/sdb2?

 Exactly same components as original RAID. sda2 and sdb2.

If you have not created a filesystem on the newly created JBOD RAID, and the RAID components was the same as in the original JBOD, your data should be safe. You only have to create the JBOD again using the same device components and, for safety, the same RAID metadata. Using the webUI will use a newer RAID metadata than the original.

No, didn't create anything above the old RAID. I assume I should not use webUI to create it again.

You should not use it to re-create the RAID, as it will use RAID metadata version 1.0. Although not very different form the original 0.9 version it's safer not to.

 

Also, JBOD creation or assembling *might* depends on the device names (sda/sdb), and this depends on whether you swapped the drives or not (left/right bay).

I took a picture of drives (with serial number) before removing them from NAS so I'm able to put them back as they were.

That is important, because device names and order play a role in JBOD; JBOD appends one disk partition after the other, creating a new md1 device where the filesystem will be laid over the start, i.e., the first component device.

In theory, JBOD uses the component devices UUID to correctly assemble the new RAID device, so device names and order should not be important.

Please post the output of the following commands, which only diagnose the current situation:

cat /proc/mdstat
mdadm --examine --scan --verbose

$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sdb1[0] sda1[1]
      524224 blocks [2/2] [UU]
     

This should be swap over RAID1, +/- 500MB. You can confirm that on the Status page, +/- 500MB of swap, and a md0 RAID1.

You have deleted the md1 device that we are going to rescue. Keep md0.

 

unused devices: <none>
$ mdadm --examine --scan --verbose
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=320150c3:466b1626:b11ebfcf:8a0c7316
   devices=/dev/sdb1,/dev/sda1

This seems to be a RAID1 created by original FW to store some data but nothing about my JBOD.

Right. swap, which is needed.

 

 

 

Thanks in advance for any help,
Axtux

Regarding those info, do you have any clue ? I'm still new to Linux so I don't know that much and I'm very afraid to do anything wrong.

EDIT : existing filesystems http://postimg.org/image/465111oil/

-sda1+sdb1 used to create a RAID1, over which swap was created and used. OK

-sda2+sdb2 used for your JBOD RAID. Each one of sda2/sb2 should be about half the 3.7TB size. Notice that ext4 appears on sdb2,  but not on sda2; sdb2 was the first JBOD disk. And you can rescue everything on sdb2 in read-only mode. If the JBOD can't be re-created you will loose all sda2 contents.

-sda3 and sdb3 created and unused by D-Link partitions. Probably for partition alignment purposes. You can latter create a fs on them and use for your own purposes.

-sda4 and sdb4 created and used by D-Link. Not used by Alt-F. You can use then for your own purposes (don't delete anything on them until you are sure you will stay with Alt-F).

webUI let me select only sda1 and sdb1 see http://postimg.org/image/vn8mpspy5/

Yes, that is an Alt-F requirement. Alt-F insists that physical partitions to be of type RAID or having already a RAID on them  to make them appear in that list. D-Link ignores partition types!

The partition type can be (later) changed to be of type RAID, that has been already addressed in this forum; just remember that you are using GPT disk partition type, use 'sgdisk' and not 'fdisk' to change the partition types.

Now, the (tentative) recovery procedure.

You might first want to mount read-only the sdb2/sda2 partition that has an ext4 filesystem on it and backup its data -- you will only save half of your data. That's the JBOD issue -- the second disk data can't be accessed, and if the first disk gets damaged you can't recover the second disk data. The reason is that the filesystem don't know that it is dealing with two disks, it only sees one big "disk" and stores files information based on that layout; if the layout changes, the fs don't find file info where it expects it to be.

The usual NO WARRANTY DISCLAIMER applies, nobody likes to play with other's people data (except Google ;-)

Assuming that md0 is your only RAID device, we will try to create a md1 JBOD RAID device.

With both disks plugged in the box, use

-the 'mount' command to be sure that none of sda2 or sdb2 are mounted.

  /dev/sda2 and /dev/sdb2 *shouldn't* appear the 'mount' output

-the 'blkid' command to know which sda2/sdb2 partitions has a ext4 fs on it, i.e., which one should be the first in the JBOD.

  The output should be /dev/sdb2: UUID=... TYPE=ext4 or /dev/sda2: UUID=... TYPE=ext4

  The one that has TYPE=ext4 should be the first in the next command. From your screen shot I will assume that it is sdb2.

-Create a new md1 JBOD RAID with metadata version 0.9 using sdb2 and sda2 as its components:

 mdadm --create /dev/md1 --run --level=linear --metadata=0.9 --raid-devices=2 /dev/sdb2 /dev/sda2

 you might get an warning that a RAID or a filesystem already exists. That *might* be leftovers from your previous attempts, go ahead.

-mount read-only the resulting md1 device and see if you can use your data:

 mkdir -p /mnt/md1

 mount -o ro /dev/md1 /mnt/md1

Your files should now be under /mnt/md1. If the mount fails or your data is not there, try reversing the sda2 sdb2 order in the 'mdadm --create' command.

Remember to try to access all files, or at least the older ones (first disk) and lots of the newer ones (second disk). Depending on old file deletion it's possible that some newer files will reside in the first disk, so make enough tests. 'find /mnt/md1' will list (without accessing its contents) all files on disk, but it will take ages...

Did it works?

All the above assumes that your first JBOD creation stored RAID info "in the usual" place in disk, only overriding the original D-Link one, and that no filesystem write or file usage has been done.

[Axtux -]

 

Either the superblock or the partition table is likely to be corrupt!

 The thing is my data was a JBOD on sda2 and sdb2, and I don't see sda2 any more.

Clarify "I don't see sda2 any more". Where don't you see it?

What I wrote is very confusing. I couldn't see sda2 any more with command line tools (it seems it displayed only filesystems, not partitions).

 

 

When you first created the JBOD, the original RAID *information* should have been erased, but only that, not the data itself. This depends on which device you said the JBOD should be created -- the same components as the original read-only one? most probably, sda2/sdb2?

 Exactly same components as original RAID. sda2 and sdb2.

If you have not created a filesystem on the newly created JBOD RAID, and the RAID components was the same as in the original JBOD, your data should be safe. You only have to create the JBOD again using the same device components and, for safety, the same RAID metadata. Using the webUI will use a newer RAID metadata than the original.

No, didn't create anything above the old RAID. I assume I should not use webUI to create it again.

You should not use it to re-create the RAID, as it will use RAID metadata version 1.0. Although not very different form the original 0.9 version it's safer not to.

Thanks for the tip, I couldn't find that on my own !

 

 

 

Thanks in advance for any help,
Axtux

Regarding those info, do you have any clue ? I'm still new to Linux so I don't know that much and I'm very afraid to do anything wrong.

EDIT : existing filesystems http://postimg.org/image/465111oil/

-sda1+sdb1 used to create a RAID1, over which swap was created and used. OK

-sda2+sdb2 used for your JBOD RAID. Each one of sda2/sb2 should be about half the 3.7TB size. Notice that ext4 appears on sdb2,  but not on sda2; sdb2 was the first JBOD disk. And you can rescue everything on sdb2 in read-only mode. If the JBOD can't be re-created you will loose all sda2 contents.

Each one of sda2/sdb2 is 3.7TB (not the half). I tried to mount sdb2 alone but I got "Invalid argument" error (it seems mount command is buggy here).
 

-sda3 and sdb3 created and unused by D-Link partitions. Probably for partition alignment purposes. You can latter create a fs on them and use for your own purposes.

-sda4 and sdb4 created and used by D-Link. Not used by Alt-F. You can use then for your own purposes (don't delete anything on them until you are sure you will stay with Alt-F).

webUI let me select only sda1 and sdb1 see http://postimg.org/image/vn8mpspy5/

Yes, that is an Alt-F requirement. Alt-F insists that physical partitions to be of type RAID or having already a RAID on them  to make them appear in that list. D-Link ignores partition types!

The partition type can be (later) changed to be of type RAID, that has been already addressed in this forum; just remember that you are using GPT disk partition type, use 'sgdisk' and not 'fdisk' to change the partition types.

Thanks for GPT tool.

Now, the (tentative) recovery procedure.

You might first want to mount read-only the sdb2/sda2 partition that has an ext4 filesystem on it and backup its data -- you will only save half of your data. That's the JBOD issue -- the second disk data can't be accessed, and if the first disk gets damaged you can't recover the second disk data. The reason is that the filesystem don't know that it is dealing with two disks, it only sees one big "disk" and stores files information based on that layout; if the layout changes, the fs don't find file info where it expects it to be.

The usual NO WARRANTY DISCLAIMER applies, nobody likes to play with other's people data (except Google ;-)

:-D

Assuming that md0 is your only RAID device, we will try to create a md1 JBOD RAID device.

With both disks plugged in the box, use

-the 'mount' command to be sure that none of sda2 or sdb2 are mounted.

  /dev/sda2 and /dev/sdb2 *shouldn't* appear the 'mount' output

-the 'blkid' command to know which sda2/sdb2 partitions has a ext4 fs on it, i.e., which one should be the first in the JBOD.

  The output should be /dev/sdb2: UUID=... TYPE=ext4 or /dev/sda2: UUID=... TYPE=ext4

  The one that has TYPE=ext4 should be the first in the next command. From your screen shot I will assume that it is sdb2.

-Create a new md1 JBOD RAID with metadata version 0.9 using sdb2 and sda2 as its components:

 mdadm --create /dev/md1 --run --level=linear --metadata=0.9 --raid-devices=2 /dev/sdb2 /dev/sda2

 you might get an warning that a RAID or a filesystem already exists. That *might* be leftovers from your previous attempts, go ahead.

-mount read-only the resulting md1 device and see if you can use your data:

 mkdir -p /mnt/md1

 mount -o ro /dev/md1 /mnt/md1

Your files should now be under /mnt/md1. If the mount fails or your data is not there, try reversing the sda2 sdb2 order in the 'mdadm --create' command.

Remember to try to access all files, or at least the older ones (first disk) and lots of the newer ones (second disk). Depending on old file deletion it's possible that some newer files will reside in the first disk, so make enough tests. 'find /mnt/md1' will list (without accessing its contents) all files on disk, but it will take ages...

Did it works?

It worked just as expected ! I wanted to do this before posting but I saw an article saying to talk about the problem before making any change to disk. I would never have thought about metadata=0.9

I would like to thank you, not only for this support but for all your work on Alt-F. I think such alternative firmware should exists to help us to be a bit more free. And your support is awesome ! Do you have a donation link ?

Regards,
Axtux

[Rolf Pedersen]

On 10/14/2015 04:06 PM, Axtux - wrote:
> I would like to thank you, not only for this support but for all your
> work on Alt-F. I think such alternative firmware should exists to help
> us to be a bit more free. And your support is awesome ! Do you have a
> donation link ?

https://sites.google.com/site/altfirmware/home/donate

>
> Regards,
> Axtux

[Axtux -]

Thanks for the link. I will make a small donation. I would like also to make a donation to your support guy Joâo Cardoso, do you have any link ?

Regards,
Axtux

[Rolf Pedersen]

That is Joâo, there, jcard, alt-f developer, support guy, the whole ball of wax!

Regards,
Axtux

[Axtux -]

I made a small donation of 10€.

While I'm at it, I've 2 other problem when I reboot my DNS-320L :
- transmission is not started though the "Boot Enabled" is checked http://postimg.org/image/rxct6gh6p/
- my /etc/inetd.conf file get 2 more lines (that seems to enable ftps) and webUI tells me to save conf because this file has changed http://postimg.org/image/rur6p3z2v/

Regards,
Axtux

[João Cardoso]

On Friday, 16 October 2015 02:29:23 UTC+1, Axtux - wrote:

I made a small donation of 10€.

Donation page updated, thanks.

 

While I'm at it, I've 2 other problem when I reboot my DNS-320L :

To keep this Topic not polluted with mixed issues you should open a new topic or continue another one.

 

- transmission is not started though the "Boot Enabled" is checked http://postimg.org/image/rxct6gh6p/

More info is needed, see a recent thread on that subject: https://groups.google.com/d/msg/alt-f/6VeLupR8t8s/60SApovTAgAJ. If more actions are needed please open a new topic with your edits and diagnostic/error messages from running the 'rctransmission' init script.

 

- my /etc/inetd.conf file get 2 more lines (that seems to enable ftps) and webUI tells me to save conf because this file has changed http://postimg.org/image/rur6p3z2v/

Yes, I think that I have that fixed now on my sources. meanwhile you will have to deal with it. That should be harmless unless it grows too much (to a few kilobytes). What Alt-F version are you running? 

 

Regards,
Axtux

[Axtux -]

 

- transmission is not started though the "Boot Enabled" is checked http://postimg.org/image/rxct6gh6p/

More info is needed, see a recent thread on that subject: https://groups.google.com/d/msg/alt-f/6VeLupR8t8s/60SApovTAgAJ. If more actions are needed please open a new topic with your edits and diagnostic/error messages from running the 'rctransmission' init script.

This topic is about transmission configuration, not start at boot. The thing is I don't know what is doing the "Boot Enabled" check so I can't diagnose it.
 

 

- my /etc/inetd.conf file get 2 more lines (that seems to enable ftps) and webUI tells me to save conf because this file has changed http://postimg.org/image/rur6p3z2v/

Yes, I think that I have that fixed now on my sources. meanwhile you will have to deal with it. That should be harmless unless it grows too much (to a few kilobytes). What Alt-F version are you running?

My ftps is disabled by default and those appended lines enable ftps, so there is a configuration change after each reboot.
 I'm using latest release 0.1RC4.1