Upgrading from 2x 1.5TB to 2x 4TB in RAID 1 configuration

[ObviousMan]

I've been reading the posts related to the drive upgrading, and although none matched the path I am on I gathered from João's responses that I can not just eject one of the 1.5TB drive, replace with 4TB one, let it sync, etc. since RAID 1 for more than 2TB requires version 1.1, and I currently have 0.9. Same post stated that I can't really do anything without breaking RAID.
It seems that the best path is to stop the device, replace both drives, let the Disk Wizard do its work on 2 new 4TB drives. After that I need a good way to transfer the data. It looks like almost any way is very time consuming.
I do have the external 1.5TB disk, and I believe I can attach it directly to the DNS-321 via USB, but it would require a "round trip" for data.
Is there a way to copy the data in the DNS from 1.5 to 4TB?
What is the best path?
Thanks.

[Rolf Pedersen]

I don't do RAID, so this might be irrelevant/bad advice, however I like gparted to copy partitions from one disk to another.  It can be got on livecd/usb or run from an installation of some Linux or other.  I've got one side of the case for one machine off and connect the two drives via sata, boot, copy/paste partition(s), one at a time, in gparted.  It's a lengthy process for big drives and this is the fastest method available to me, alternately using esata. 

[João Cardoso]

On Tuesday, 26 April 2016 14:07:24 UTC+1, ObviousMan wrote:

I've been reading the posts related to the drive upgrading, and although none matched the path I am on I gathered from João's responses that I can not just eject one of the 1.5TB drive, replace with 4TB one, let it sync, etc. since RAID 1 for more than 2TB requires version 1.1, and I currently have 0.9. Same post stated that I can't really do anything without breaking RAID.

That is not exactly correct, there is a way (posted by mdadm author) to convert raid metadata version 0.9 to 1.0 (*not* 1.1 or 1.2), see https://raid.wiki.kernel.org/index.php/RAID_superblock_formats#Converting_between_superblock_versions.

I think to remember to have successfully tried that.

Accordingly to the same site

The version-0.90 superblock (...) limits each component device to a maximum size of 2TB on kernel version <3.1 and 4TB on kernel version >=3.1.

As rc4.1 uses linux 3.10, metadata version 0.9 should be OK for 4TB components. I have not verified this. 

So it is possibly to convert to 1.0, and do the 1.5 to 4TB upgrade:

-fail one component, remove that component disk, paper label it and keep it in a drawer as a backup

-convert partition tables from MBR to GPT if needed (*)

-convert to metadata 1.0 (seems to be optional)

-add one new 4TB drive following normal raid 1 degraded fix procedure, wait for resync to finish

-fail and remove the other 1.5TB disk and repeat the previous step

-stop the raid

-re-partition (*) both 4TB disks to increase its raid partition to use the full disk (*don't* change the raid partition start, only its length)

-start the raid

-enlarge the raid,

-enlarge the filesystem

(*)-repartition might not be possible using the webUI, depending on the partition scheme. D-Link uses out-of-order sectors partitioning, and the conversion from MBR to GPT might fail

So, it's complex, time-lengthy, and subjected to a few peculiarities that depends on your particular setup.

 

It seems that the best path is to stop the device, replace both drives, let the Disk Wizard do its work on 2 new 4TB drives. After that I need a good way to transfer the data. It looks like almost any way is very time consuming.
I do have the external 1.5TB disk, and I believe I can attach it directly to the DNS-321 via USB, but it would require a "round trip" for data.

Disk to disk copy on the box is lengthy. And using USB worses that.

Is there a way to copy the data in the DNS from 1.5 to 4TB?

-fail the existing raid and remove the failed disk (paper label it and save it as a backup in a drawer)

-insert  one of the 4TB disks and create a *new* degraded raid with it

-copy from old to new raid (1-3 days?), stop old, remove it, add 2nd 4TB, add it to the new raid.

Ready to use, but the raid rsync will take another couple of days to finish.

also lengthy, but the copy will be faster than using USB, and it will not be dependant on your particular partition setup. Easier, also.

What is the best path?

 

The faster (5x ?) copy path would be to attach one of the 1.5TB and one of the Disk Wizard setup  4TB disks on a PC and do the copy there.

The disks could be used in the PC either as degraded raid or mounted as normal non-RAID disks; that is a 0.9/1.0 raid metadata feature if used properly, but you couldn't use Rolf gparted copy suggestion. You can however use his linux live usb suggestion if you don't use linux on your PC.

drawback: use advanced linux command line. No free launch.

 

Thanks.

[ObviousMan]

João,
Thank you for your suggestions. Upgrading the superblock seem a bit dangerous, as it appears easy to mess up the command line procedure. If 0.9 would allow me to use 4TB (just grabbed a pair of WD off Newegg) as the wiki suggests, I am fine with that.
Seems like this is the simplest and least problem prone procedure quoted from your response:
==<Quote>==

1. fail one drive in the existing raid
2. remove the failed disk (paper label it and save it as a backup in a drawer)

3. insert  one of the 4TB disks and create a *new* degraded raid with it (I presume use Disk Wizard? or more likely manually?)

4. copy from old to new raid (1-3 days?),
5. stop old, remove it.
6. add 2nd 4TB, add it to the new raid.
==<End Quote>==

When the new degraded RAID is create on new 4TB disk, if Disk Wizard is used (or manually - I am a bit hazy on this part), is it going to create the 1.1 superblock?
Thank you for your help.
OM

[João Cardoso]

On Thursday, 28 April 2016 01:38:23 UTC+1, ObviousMan wrote:

João,
Thank you for your suggestions. Upgrading the superblock seem a bit dangerous, as it appears easy to mess up the command line procedure. If 0.9 would allow me to use 4TB (just grabbed a pair of WD off Newegg) as the wiki suggests, I am fine with that.
Seems like this is the simplest and least problem prone procedure quoted from your response:
==<Quote>==

1. fail one drive in the existing raid
2. remove the failed disk (paper label it and save it as a backup in a drawer)

3. insert  one of the 4TB disks and create a *new* degraded raid with it (I presume use Disk Wizard? or more likely manually?)

 

Try first the Disk Wizard. I know that it fails for 6TB disks, not sure if 4TB will work, there are mixed reports.

If that fail you can use the command line, a couple of simple commands will do it (they have already been posted, not sure where, please report back and I will repost them) 

 

4. copy from old to new raid (1-3 days?),
5. stop old, remove it.
6. add 2nd 4TB, add it to the new raid.

 

before adding it to the raid it must be partitioned, copying (Disk Partitioner or using the cmd line) the partition table of the existing 4TB is the easiest way.

==<End Quote>==

When the new degraded RAID is create on new 4TB disk, if Disk Wizard is used (or manually - I am a bit hazy on this part), is it going to create the 1.1 superblock?

No, for rc4.1 webUI it will be 1.0.

Both 0.9 and 1.0 are located at the partition end, which makes it possible to use the filesystem directly without using raid; 1.1 and 1.2 are located at the partition start, and require assembling raid first before being able to use the fs. Each has its advantages and inconveniences, 1.0 has the added benefice of being "compatible" with 0.9.

Unfortunately, the default metadata used when creating a new raid from the cmd line and no metadata version is specified is 1.2, which is a non-compatible departure from the default.

[ObviousMan]

Thank you for clarifications. I will give it a shot over the weekend. The drives just got here.

[ObviousMan]

Well, Disk Wizard does fail.
I've tried to look up the right sequence of commands (or may be just one) to create degraded RAID1 on the new disk, but it is a bit confusing.
Do I need to format  first, then

mdadm --create /dev/md0 -l6 -n8 -c64 --layout=la --level=raid6 --raid-devices=2 --metadata=1.0 --assume-clean /dev/sda1 missing
Replace /dev/md0 and /dev/sda1 appropriately.

Or I confused myself?
Also I do not seem to be able to start long test...

Thanks.
OM

[ObviousMan]

I forgot to mention that I have 2 other partitions on the currently degraded old RAID 1:
=========
   Device Boot      Start         End      Blocks  Id System
/dev/sdb1               1          66      530113+ 82 Linux swap
/dev/sdb2             131      182236  1462766445  83 Linux
/dev/sdb4              67         130      514080  83 Linux
========
What will happen with swap (one partition), and how do I manually allocate the partition where /Users and /Public are located (do I just copy it first).
I am starting to confuse myself again...
Thanks.
OM

[João Cardoso]

On Sunday, 1 May 2016 04:06:30 UTC+1, ObviousMan wrote:

Well, Disk Wizard does fail.

Then you have to use the Disk Partitioner (or the cmd line) to create a 512MB swap-type partition in the 1st partition and a 2nd raid-type partition occupying all remaining space. On the *new* disk!

 

I've tried to look up the right sequence of commands (or may be just one) to create degraded RAID1 on the new disk, but it is a bit confusing.

After partitioning you can use the raid webui to create the new degraded raid1. be sure to *not* select as component the device partition (component)  used by the existing degraded raid1.

 

Do I need to format  first, then

mdadm --create /dev/md0 -l6 -n8 -c64 --layout=la --level=raid6 --raid-devices=2 --metadata=1.0 --assume-clean /dev/sda1 missing

mdadm --create /dev/mdN --run --chunk=512 --bitmap=internal --level=raid1 --raid-devices=2 --metadata=1.0 /dev/sdXM missing # all on a single line

posting a screenshot of your current Status, RAID, Partitioner (for the two disks) webui would help clarify the N, X, M above

 

Replace /dev/md0 and /dev/sda1 appropriately.

Or I confused myself?
Also I do not seem to be able to start long test...

you better describe, in sequence, what you have done till now. That will avoid misunderstandings.

[ObviousMan]

On Sunday, May 1, 2016 at 10:17:57 AM UTC-5, João Cardoso wrote:

On Sunday, 1 May 2016 04:06:30 UTC+1, ObviousMan wrote:

Well, Disk Wizard does fail.

Then you have to use the Disk Partitioner (or the cmd line) to create a 512MB swap-type partition in the 1st partition and a 2nd raid-type partition occupying all remaining space. On the *new* disk!

I have done nothing on the new drive yet besides sticking it into the device.

I have the swap partition on the existing drive, and also a small ~500MB partition with /Users /Public directories on the drive. I am thinking of just copying them to the new drive, as they are the first two partitions.
 

 

I've tried to look up the right sequence of commands (or may be just one) to create degraded RAID1 on the new disk, but it is a bit confusing.

After partitioning you can use the raid webui to create the new degraded raid1. be sure to *not* select as component the device partition (component)  used by the existing degraded raid1.

 

Do I need to format  first, then

mdadm --create /dev/md0 -l6 -n8 -c64 --layout=la --level=raid6 --raid-devices=2 --metadata=1.0 --assume-clean /dev/sda1 missing

mdadm --create /dev/mdN --run --chunk=512 --bitmap=internal --level=raid1 --raid-devices=2 --metadata=1.0 /dev/sdXM missing # all on a single line

posting a screenshot of your current Status, RAID, Partitioner (for the two disks) webui would help clarify the N, X, M above

I can figure N, X, M (I think) - current degraded RAID is md0 on partition /dev/sdb4, so new one will be md1, new drive is listed as /dev/sdc, so probably RAID will be on /dev/sdc4
In any case, I have attached the screen shots
 

 

Replace /dev/md0 and /dev/sda1 appropriately.

Or I confused myself?
Also I do not seem to be able to start long test...

you better describe, in sequence, what you have done till now. That will avoid misunderstandings.

Besides failing the /dev/sda, taking it out, and sticking the new WD drive into the device, I have not done anything yet

[João Cardoso]

On Sunday, 1 May 2016 04:24:14 UTC+1, ObviousMan wrote:

I forgot to mention that I have 2 other partitions on the currently degraded old RAID 1:

You should not touch anything on the degraded raid1 containing disk. Better start using terms like 1.5 and 4TB disks.

 

=========
   Device Boot      Start         End      Blocks  Id System
/dev/sdb1               1          66      530113+ 82 Linux swap
/dev/sdb2             131      182236  1462766445  83 Linux
/dev/sdb4              67         130      514080  83 Linux
========

You still have the D-Link partition layout on the 1.5 disk. sdb2 is the component of your degraded raid1 (probably called md0). you don't have to do anything on the 1.5 disk.

 

What will happen with swap (one partition), and how do I manually allocate the partition where /Users and /Public are located (do I just copy it first).

After having the new degraded raid1 on the 4TB disk created, it will have a name such as md1 (if md0 is the name of the degraded 1.5 raid1). Then you have to create a (ext4) filesystem on that (md1?) device (Disk->Filesystem). Then you have to create and activate the swap on the 4 disk. All that (and partitioning) is what the Wizard does and now you have to do step by step.

Then you have to copy all data from from the 1.5 raid1 filesystem to the 4 raid1 filesystem. That will copy the Users and Public

 

I am starting to confuse myself again...

lets proceed step by step.

-You have failed the 1.5 raid1 setup, removed one of the 1.5 disks and plugged one of the 4 disks.

(meanwhile I have received your screenshots)

-you now have to partition sdc, using the Partitioner on the 4TB disk, with:

 --a 512MB swap-type partition on the 1st partition (it will be called sdc1)

 --a 2nd raid-type partition (sdc2) occupying all remaining space (you can leave a small amount of free space)

 --optional: if you have put and are using anything on sdb4 on the 1.5 disk, create a 3d 512MB partition linux-type (sdc3)

-then create (Raid webui) a degraded raid1 using sdc2, it will be called md1

-then create a ext4 filesystem (Filesystem webui) on md1

-then create a ext4 filesystem (Filesystem webui) on sdc3 (if it applies)

I will wait for your completion feedback before continuing.

NOTICE: Be aware that the 4T disk called sdc (and all its sdcN partitions) will most probably be called sda (and sda1, etc) after a reboot!

So if you do a reboot, please refer that and post/refer all the new device names/screenshots otherwise my indications will be wrong

[João Cardoso]

On Sunday, 1 May 2016 17:06:22 UTC+1, João Cardoso wrote:

On Sunday, 1 May 2016 04:24:14 UTC+1, ObviousMan wrote:

I forgot to mention that I have 2 other partitions on the currently degraded old RAID 1:

You should not touch anything on the degraded raid1 containing disk. Better start using terms like 1.5 and 4TB disks. 

... 

You still have the D-Link partition layout on the 1.5 disk. sdb2 is the component of your degraded raid1 (probably called md0). you don't have to do anything on the 1.5 disk.

 

What will happen with swap (one partition), and how do I manually allocate the partition where /Users and /Public are located (do I just copy it first).

After having the new degraded raid1 on the 4TB disk created, it will have a name such as md1 (if md0 is the name of the degraded 1.5 raid1). Then you have to create a (ext4) filesystem on that (md1?) device (Disk->Filesystem). Then you have to create and activate the swap on the 4 disk. All that (and partitioning) is what the Wizard does and now you have to do step by step.

Then you have to copy all data from from the 1.5 raid1 filesystem to the 4 raid1 filesystem. That will copy the Users and Public

 

I am starting to confuse myself again...

lets proceed step by step.

-You have failed the 1.5 raid1 setup, removed one of the 1.5 disks and plugged one of the 4 disks.

(meanwhile I have received your screenshots)

-you now have to partition sdc, using the Partitioner on the 4TB disk, with:

I just noticed something wrong in your Partitioner screenshot:

-no "Partition" button (what browser are you using?)

-"Using MBR partitioning" message

-The former might be a browser issue, try Chrome or Firefox

-The latter is a bug and can probably be circumvented by creating any partition, any size/type, and then using the top section to "Convert" the MBR to GPT partition on the 4T disk

After that succeeds, the message will be "Using GPT partitioning", as it has to be for bigger than 2TB disks, and you can overwrite the above created dummy partition continuing with the following

[ObviousMan]

Ok, I made a wild guess, and thought that if there is an issue with Partitioner in IE, perhaps the Disc Wizard would work  in Chrome.
So before I did anything else, I went into Disk Wizard, unselected /dev/sdb, selected ext 4 and raid 1, and "abracadabra"
And guess what? It did, at least partially, kind of...
I attached the screen shot of disk wizard. It has been stuck in this state for last half an hour or so.
The device would not accept telnet in, or another HTTP connection. The HD lights on device are not blinking at all, I watched for a couple minutes straight.
I wanted to check how long does formatting step should take for a 4TB drive? I do not want to do anything drastic in case it is still busy formatting..
Also I noticed that the fan on device is not running at all
Thanks again.
OM

On Sunday, May 1, 2016 at 11:36:00 AM UTC-5, João Cardoso wrote:

On Sunday, 1 May 2016 17:06:22 UTC+1, João Cardoso wrote:

You should not touch anything on the degraded raid1 containing disk. Better start using terms like 1.5 and 4TB disks. 

...

[João Cardoso]

On Sunday, 1 May 2016 20:16:46 UTC+1, ObviousMan wrote:

Ok, I made a wild guess, and thought that if there is an issue with Partitioner in IE,

I had a  top-posted forum message saying that there are issues with IE and I don't test Alt-F with it. I have to resurrect that post ;-)

perhaps the Disc Wizard would work  in Chrome.
So before I did anything else, I went into Disk Wizard, unselected /dev/sdb, selected ext 4 and raid 1, and "abracadabra"
And guess what? It did, at least partially, kind of...
I attached the screen shot of disk wizard. It has been stuck in this state for last half an hour or so.

 

The device would not accept telnet in, or another HTTP connection. The HD lights on device are not blinking at all, I watched for a couple minutes straight.
I wanted to check how long does formatting step should take for a 4TB drive?

it takes a couple of hours, I *guess*, but the drive leds should flash and you should be able to telnet or ssh the box and see the last message character in the browser iterate through -\|/, simulating a moving pattern.

 

I do not want to do anything drastic in case it is still busy formatting..

I never used a DNS-321 and don't remember if anyone reported on how the power button normally works on it, but you must know (and tell us?).

If the disk leds don't blink (and they usually bink on disk access -- again I never used a 321 and the leds behaviour is different on different boxes), then the box is frozen and you can try to reboot/power-down using the power button. In last case pull the power cord.

The good news is that all partitioning, swap and raid creation are now done, only the filesystem step (eventually) failed. So, if it does not complete, at the next reboot you will see a error/warning message regarding 'fsck' on that new 'md0' or 'md1' filesystem and you have to "Create" (not "Check") one using the "Filesystem" webui.

Recheck the devices names before doing that, not only sda/sdb/sdc have changed but md0/md1 might also have changed.

Also I noticed that the fan on device is not running at all

If there is no disk activity (you can also try to ear/finger-feel them spinning), then no heat is being developed, no temperature rise and the fan might have been stopped deliberately. Only you know what is/how-you-set the normal fan behaviour.

[ObviousMan]

On Sunday, May 1, 2016 at 3:31:36 PM UTC-5, João Cardoso wrote:

I had a  top-posted forum message saying that there are issues with IE and I don't test Alt-F with it. I have to resurrect that post ;-)

I missed that one before. Sorry.

it takes a couple of hours, I *guess*, but the drive leds should flash and you should be able to telnet or ssh the box and see the last message character in the browser iterate through -\|/, simulating a moving pattern.

I waited for just over 2 hours, there never was \|/. Power button would do nothing. Had to do good old fashioned power supply cable pull.

 

I never used a DNS-321 and don't remember if anyone reported on how the power button normally works on it, but you must know (and tell us?).

If the disk leds don't blink (and they usually bink on disk access -- again I never used a 321 and the leds behaviour is different on different boxes), then the box is frozen and you can try to reboot/power-down using the power button. In last case pull the power cord.

The good news is that all partitioning, swap and raid creation are now done, only the filesystem step (eventually) failed. So, if it does not complete, at the next reboot you will see a error/warning message regarding 'fsck' on that new 'md0' or 'md1' filesystem and you have to "Create" (not "Check") one using the "Filesystem" webui.

Recheck the devices names before doing that, not only sda/sdb/sdc have changed but md0/md1 might also have changed.

I am attaching the screen shots of status, partitioner, filesystem, raid.
As suggested I am formatting md1 as ext4. How long that should take? It came back fairly soon. Now is showing ext4 filesystem on md1.

Also I noticed that the fan on device is not running at all

If there is no disk activity (you can also try to ear/finger-feel them spinning), then no heat is being developed, no temperature rise and the fan might have been stopped deliberately. Only you know what is/how-you-set the normal fan behaviour.

Fun started running on re-start for a while. My guess the drives were completely off-line, so fan did not need to run, as you supposed.
 

[Joao Cardoso]

On May 1, 2016 10:24 PM, "ObviousMan" <bre...@gmail.com> wrote:
>
>
>
> On Sunday, May 1, 2016 at 3:31:36 PM UTC-5, João Cardoso wrote:
>>
>>
>> I had a  top-posted forum message saying that there are issues with IE and I don't test Alt-F with it. I have to resurrect that post ;-)
>>
> I missed that one before. Sorry.
>>
>>
>> it takes a couple of hours, I *guess*, but the drive leds should flash and you should be able to telnet or ssh the box and see the last message character in the browser iterate through -\|/, simulating a moving pattern.
>
> I waited for just over 2 hours, there never was \|/. Power button would do nothing. Had to do good old fashioned power supply cable pull.
>>
>>  
>> I never used a DNS-321 and don't remember if anyone reported on how the power button normally works on it, but you must know (and tell us?).
>> If the disk leds don't blink (and they usually bink on disk access -- again I never used a 321 and the leds behaviour is different on different boxes), then the box is frozen and you can try to reboot/power-down using the power button. In last case pull the power cord.
>>
>> The good news is that all partitioning, swap and raid creation are now done, only the filesystem step (eventually) failed. So, if it does not complete, at the next reboot you will see a error/warning message regarding 'fsck' on that new 'md0' or 'md1' filesystem and you have to "Create" (not "Check") one using the "Filesystem" webui.
>> Recheck the devices names before doing that, not only sda/sdb/sdc have changed but md0/md1 might also have changed.
>
> I am attaching the screen shots of status, partitioner, filesystem, raid.

All looks ok

> As suggested I am formatting md1 as ext4. How long that should take?

Don't know. Are the leds blinking?

> It came back fairly soon. Now is showing ext4 filesystem on md1.

If the status and filesystem page  displays ext4 on md1 it has finished and ready to go

You can now copy everything from md0 to md1.

Although the Folder Browse webui has a copy button I don't recomend using it on this, it will only significantely slow down the copy.

The command
'cp -a /mnt/md0/ /mnt/md1' does it faster (~10MB/s, do your math), but without any progress indication. Use the status page to see the md1 fs size increasing. Don't check the Refresh checkbox, it will be slower.

>>
>>
>>> Also I noticed that the fan on device is not running at all
>>
>>
>> If there is no disk activity (you can also try to ear/finger-feel them spinning), then no heat is being developed, no temperature rise and the fan might have been stopped deliberately. Only you know what is/how-you-set the normal fan behaviour.
>
> Fun started running on re-start for a while. My guess the drives were completely off-line, so fan did not need to run, as you supposed.
>  
>>
>> On Sunday, May 1, 2016 at 11:36:00 AM UTC-5, João Cardoso wrote:
>>>>
>>>>
>>>>
>>>> On Sunday, 1 May 2016 17:06:22 UTC+1, João Cardoso wrote:
>>>>>
>>>>>
>>>>> You should not touch anything on the degraded raid1 containing disk. Better start using terms like 1.5 and 4TB disks. 
>>>>
>>>> ...
>>>>
>>>> I just noticed something wrong in your Partitioner screenshot:
>>>> -no "Partition" button (what browser are you using?)
>>>> -"Using MBR partitioning" message
>>>>
>>>> -The former might be a browser issue, try Chrome or Firefox
>>>> -The latter is a bug and can probably be circumvented by creating any partition, any size/type, and then using the top section to "Convert" the MBR to GPT partition on the 4T disk
>>>> After that succeeds, the message will be "Using GPT partitioning", as it has to be for bigger than 2TB disks, and you can overwrite the above created dummy partition continuing with the following
>>>>  
>>>>>
>>>>>  --a 512MB swap-type partition on the 1st partition (it will be called sdc1)
>>>>>  --a 2nd raid-type partition (sdc2) occupying all remaining space (you can leave a small amount of free space)
>>>>>  --optional: if you have put and are using anything on sdb4 on the 1.5 disk, create a 3d 512MB partition linux-type (sdc3)
>>>>> -then create (Raid webui) a degraded raid1 using sdc2, it will be called md1
>>>>> -then create a ext4 filesystem (Filesystem webui) on md1
>>>>> -then create a ext4 filesystem (Filesystem webui) on sdc3 (if it applies)
>>>>>
>>>>> I will wait for your completion feedback before continuing.
>>>>>
>>>>> NOTICE: Be aware that the 4T disk called sdc (and all its sdcN partitions) will most probably be called sda (and sda1, etc) after a reboot!
>>>>> So if you do a reboot, please refer that and post/refer all the new device names/screenshots otherwise my indications will be wrong
>

> --
> You received this message because you are subscribed to the Google Groups "Alt-F" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to alt-f+un...@googlegroups.com.
> Visit this group at https://groups.google.com/group/alt-f.
> For more options, visit https://groups.google.com/d/optout.

[ObviousMan]

On Sunday, May 1, 2016 at 4:59:11 PM UTC-5, João Cardoso wrote:

You can now copy everything from md0 to md1.

Although the Folder Browse webui has a copy button I don't recomend using it on this, it will only significantely slow down the copy.

The command
'cp -a /mnt/md0/ /mnt/md1' does it faster (~10MB/s, do your math), but without any progress indication. Use the status page to see the md1 fs size increasing. Don't check the Refresh checkbox, it will be slower.

I did check the file system before I started copying, it took close to 1.5 hours.
Started copying via cp -a. I guess this will take more than a day.
>EDIT> Strangely it finished by morning. I had to leave for work without doing anything else. I wonder if it failed along the way in some way that did not generate error message. What is the simplest way to compare at least the directory structure? "Cat" the recursive "ls" into the files and "diff" them?
OM

[ObviousMan]

Well, it is all completely screwed up now. Apparently the system has been residing on /mnt/sdb4 - the other partition on the original disk
After I verified the copied data, I stopped the original disk and removed it. That is where all broke down.
I put the original disk back in and restarted - it does not respond to any commands, the WebUI does not respond, telnet does not respond. I though most resided on internal memory, but somehow it is all broken now.
It is not even active on the network in any shape or form. The router does not see the box as connected.
I understand the data on the HD's is fine, but how do I ressurect the box now?
OM

[João Cardoso]

On Tuesday, 3 May 2016 02:34:59 UTC+1, ObviousMan wrote:

Well, it is all completely screwed up now.

that's not enough information

Apparently the system has been residing on /mnt/sdb4

No, only non-essential disk-installable packages might be there. Alt-F is stored in non-volatile flash memory and works perfectly without any drive plugged.

- the other partition on the original disk
After I verified the copied data, I stopped the original disk and removed it.

And *how* exactly did you do that? It's not easy if Alt-F packages were installed and running in that disk. The webui would complain in that case.

That is where all broke down.
I put the original disk back in and restarted

try first without any disk plugged in. Most probably the box IP has changed.

 

- it does not respond to any commands, the WebUI does not respond, telnet does not respond. I though most resided on internal memory, but somehow it is all broken now.
It is not even active on the network in any shape or form. The router does not see the box as connected.

or the box is using a static IP. network cables?

do network leds turn-in in both the router and the box? do they blink? In the 323 the network led is in the front, between the two drive leds, but you said in the first post that your box is a 321.

I understand the data on the HD's is fine, but how do I ressurect the box now?

is the box alive? read the "about leds and buttons" wiki and tell us the results.

are network cables OK? boot with no drives in the box, see if the router sees it.

Still nothing? try the "long reset" procedure.

There is nothing in the 2x1.5 to 2x4TB raid1 upgrade procedure that we have been talking about that could brick the box!

or it is not bricked and you can't access it or as unfortunate coincidence the box decided to fail now.

[ObviousMan]

On Tuesday, May 3, 2016 at 9:58:11 AM UTC-5, João Cardoso wrote:

On Tuesday, 3 May 2016 02:34:59 UTC+1, ObviousMan wrote:

Well, it is all completely screwed up now.

that's not enough information

I agree

Apparently the system has been residing on /mnt/sdb4

No, only non-essential disk-installable packages might be there. Alt-F is stored in non-volatile flash memory and works perfectly without any drive plugged.

Then it is really strange.

- the other partition on the original disk
After I verified the copied data, I stopped the original disk and removed it.

And *how* exactly did you do that? It's not easy if Alt-F packages were installed and running in that disk. The webui would complain in that case.

I did it as you suggested in your procedure. I pressed the "stop" button in the  WebUI. It showed "stopped", so I removed the original hd from the case.
WebUI did not complain at that point.

That is where all broke down.
I put the original disk back in and restarted

try first without any disk plugged in. Most probably the box IP has changed.

It is possible, but I do not see it on the network at all. Network mapper on the router does not see it being connected at all (Tomato) I will try with no disks. 

 

- it does not respond to any commands, the WebUI does not respond, telnet does not respond. I though most resided on internal memory, but somehow it is all broken now.
It is not even active on the network in any shape or form. The router does not see the box as connected.

or the box is using a static IP. network cables?

do network leds turn-in in both the router and the box? do they blink? In the 323 the network led is in the front, between the two drive leds, but you said in the first post that your box is a 321.

It is 323. I must have had the brain fart when I posted the 1st one.

It was using static address. Network led would turn on on the box, blink for a while, and then go away. Network cable did not change, but I can try a different one.

I understand the data on the HD's is fine, but how do I ressurect the box now?

is the box alive? read the "about leds and buttons" wiki and tell us the results.

are network cables OK? boot with no drives in the box, see if the router sees it.

Still nothing? try the "long reset" procedure.

Will do that.

There is nothing in the 2x1.5 to 2x4TB raid1 upgrade procedure that we have been talking about that could brick the box!

or it is not bricked and you can't access it or as unfortunate coincidence the box decided to fail now.

I did not think so either. May be just exercising the box more than regular did something.

Sorry, I got a bit panicked. I will follow the advise and report back after I am back from work.

[ObviousMan]

Just to confirm, the box is DNS-323 Rev A.

[João Cardoso]

On Tuesday, 3 May 2016 17:35:15 UTC+1, ObviousMan wrote:

Just to confirm, the box is DNS-323 Rev A.

OK. The clues I gave previously still apply.

Just some additional notes: 

-a lamp works until if fails; the same happens with a network cable...

-Even without network access, you can verify if Alt-F is working by pressing (and keep pressing) the front power button and watching the orange disk leds.  If they blink at a time (see the wiki) then Alt-F is working

-to force DHCP instead of a static IP (after verifying the network cables), the longer than 20 seconds reset button press will clear all settings to the defaults.

-you should do all these tests without disks attached, to save the disks from unnecessary stress and to guarantee that settings eventually stored on disk (under the Alt-F folder) don't override the box defaults

-the right way to remove a disk is to "Eject" it using the Disk Maintenance webui; stopping a raid device is not enough, as a disk can have active/mounted filesystems on other disk partitions. And swap might be in use. "Eject" on Disk Utilities safely stops them in the right order, but if Alt-F disk installed packages are in any disk filesystem the eject will most likely fail.

[ObviousMan]

On Tuesday, May 3, 2016 at 3:19:34 PM UTC-5, João Cardoso wrote:

Just some additional notes: 

-a lamp works until if fails; the same happens with a network cable...

I tried two other cables, one for sure working, the other is new, straight from Monoprice.

-Even without network access, you can verify if Alt-F is working by pressing (and keep pressing) the front power button and watching the orange disk leds.  If they blink at a time (see the wiki) then Alt-F is working

Each orange LEDs blinked 4 times

-to force DHCP instead of a static IP (after verifying the network cables), the longer than 20 seconds reset button press will clear all settings to the defaults.

I am going to do that, as I suspect somehow the device may be on a different subnet, so I do not see it on the network map.

-you should do all these tests without disks attached, to save the disks from unnecessary stress and to guarantee that settings eventually stored on disk (under the Alt-F folder) don't override the box defaults

Yes, took both disks out yesterday.

-the right way to remove a disk is to "Eject" it using the Disk Maintenance webui; stopping a raid device is not enough, as a disk can have active/mounted filesystems on other disk partitions. And swap might be in use. "Eject" on Disk Utilities safely stops them in the right order, but if Alt-F disk installed packages are in any disk filesystem the eject will most likely fail.

 I did not know that one. Your directions said "stop", I took it literally...
OM

[João Cardoso]

On Wednesday, 4 May 2016 02:29:03 UTC+1, ObviousMan wrote:

On Tuesday, May 3, 2016 at 3:19:34 PM UTC-5, João Cardoso wrote:

Just some additional notes: 

-a lamp works until if fails; the same happens with a network cable...

I tried two other cables, one for sure working, the other is new, straight from Monoprice.

-Even without network access, you can verify if Alt-F is working by pressing (and keep pressing) the front power button and watching the orange disk leds.  If they blink at a time (see the wiki) then Alt-F is working

Each orange LEDs blinked 4 times

so Alt-F is working, it is the 'sysctrl' process that reads buttons and activates leds.

the box is not bricked and if there is any network issue Alt-F assigned itself the 192.168.1.254, if used then 192.168.1.253, ... IP address.

You can try a direct PC-box network cable connection. that has been addressed several times in the forum

 

-to force DHCP instead of a static IP (after verifying the network cables), the longer than 20 seconds reset button press will clear all settings to the defaults.

I am going to do that, as I suspect somehow the device may be on a different subnet, so I do not see it on the network map.

-you should do all these tests without disks attached, to save the disks from unnecessary stress and to guarantee that settings eventually stored on disk (under the Alt-F folder) don't override the box defaults

Yes, took both disks out yesterday.

-the right way to remove a disk is to "Eject" it using the Disk Maintenance webui; stopping a raid device is not enough, as a disk can have active/mounted filesystems on other disk partitions. And swap might be in use. "Eject" on Disk Utilities safely stops them in the right order, but if Alt-F disk installed packages are in any disk filesystem the eject will most likely fail.

 I did not know that one. Your directions said "stop", I took it literally...

OK, I might have been inaccurate. "stop" the disk, not  just the raid (there might exist other raids on the disk, mounted filesystems, swap, etc. An ordered shutdown and a proper startup was the main motivation for Alt-F)

but I don't see how anything in the disk upgrade procedure could "brick" or change the box IP.

OM

[ObviousMan]

On Wednesday, May 4, 2016 at 11:32:34 AM UTC-5, João Cardoso wrote:

so Alt-F is working, it is the 'sysctrl' process that reads buttons and activates leds.

the box is not bricked and if there is any network issue Alt-F assigned itself the 192.168.1.254, if used then 192.168.1.253, ... IP address.

You can try a direct PC-box network cable connection. that has been addressed several times in the forum

OK, I might have been inaccurate. "stop" the disk, not  just the raid (there might exist other raids on the disk, mounted filesystems, swap, etc. An ordered shutdown and a proper startup was the main motivation for Alt-F)

but I don't see how anything in the disk upgrade procedure could "brick" or change the box IP.

It is working, although it was not on 192.168.1.254 or 253. I used the long press reset as you earlier suggested, the box showed up on the network after it booted up.
It came up with the 1st login wizard, etc but it would not save settings until I inserted the drive. I inserted the new 4TB drive where it was before.
The wizard wanted me to create the user directories, I allowed it to do that in md1, as it was the only available. After a short period of time the error message came up:

Unable to automatically fix md1, mounting Read Only: fsck 1.41.14 (22-Dec-2010)
/dev/md1 is mounted.  e2fsck: Cannot continue, aborting.

At this point I really do not want to mess anything up, so please suggest what the next action should be.
I have not tried
I am attaching the screen shots.
Thank you.
OM

[João Cardoso]

On Wednesday, 4 May 2016 19:01:51 UTC+1, ObviousMan wrote:

On Wednesday, May 4, 2016 at 11:32:34 AM UTC-5, João Cardoso wrote:

so Alt-F is working, it is the 'sysctrl' process that reads buttons and activates leds.

the box is not bricked and if there is any network issue Alt-F assigned itself the 192.168.1.254, if used then 192.168.1.253, ... IP address.

You can try a direct PC-box network cable connection. that has been addressed several times in the forum

OK, I might have been inaccurate. "stop" the disk, not  just the raid (there might exist other raids on the disk, mounted filesystems, swap, etc. An ordered shutdown and a proper startup was the main motivation for Alt-F)

but I don't see how anything in the disk upgrade procedure could "brick" or change the box IP.

It is working, although it was not on 192.168.1.254 or 253. I used the long press reset as you earlier suggested, the box showed up on the network after it booted up.
It came up with the 1st login wizard,

Yes, the long reset clears all settings, that has already been covered.

 

etc but it would not save settings until I inserted the drive.

That is not expected. What steps did you take? any error or warning message?

 

I inserted the new 4TB drive where it was before.
The wizard wanted me to create the user directories, I allowed it to do that in md1, as it was the only available. After a short period of time the error message came up:

Unable to automatically fix md1, mounting Read Only: fsck 1.41.14 (22-Dec-2010)
/dev/md1 is mounted.  e2fsck: Cannot continue, aborting.

At this point I really do not want to mess anything up, so please suggest what the next action should be.

Filesystems->FS Operations->Force Fix. *read* the popup warning but search first the forum with the error message:  "Unable to automatically fix mounting Read Only" for details.

That should have been deployed by an unclean shutdown

I have not tried
I am attaching the screen shots.

everything else looks OK. You have swap, needed by fsck, a working degraded raid, and a working (but with errors) filesystem

 

Thank you.
OM

[ObviousMan]

On Thursday, May 5, 2016 at 11:13:07 AM UTC-5, João Cardoso wrote:

but I don't see how anything in the disk upgrade procedure could "brick" or change the box IP.

This bit is strange to me as well.
 

etc but it would not save settings until I inserted the drive.

That is not expected. What steps did you take? any error or warning message?

I just inserted the drive. It failed to save, I did not see messages,

 

Filesystems->FS Operations->Force Fix. *read* the popup warning but search first the forum with the error message:  "Unable to automatically fix mounting Read Only" for details.

I did go there almost after I posted the reply. Actually I tried to run check again, and it was fine, I did not neet do Force Fix.

That should have been deployed by an unclean shutdown

I figured that too.

everything else looks OK. You have swap, needed by fsck, a working degraded raid, and a working (but with errors) filesystem

I did not have any time to touch the device yesterday, but earlier today I followed your earlier suggestion on inserting the 2nd 4TB disk (done), copying the partition table from 1st 4TB disk to it (done), and then adding the new filesystem to the previously degraded RAID (done).
It is now in "recover" mode, and will be in such for another 8+ hours if you believe the time estimate. My guess it is back on track.

I just want to make sure there are no elements of Alt-F, etc. are missing, so I wanted to ask what else I can check to assure "long term health".
Thank you again for your help.
OM

[João Cardoso]

The Alt-F folder only contain files from new or updated Alt-F packages (Packages->Alt->F), so if you don't need any that folder is not needed and might even cause some annoyances. *Always* use the Packages webui to manipulate the folder, or the box might froze -- read its README.txt.

 

so I wanted to ask what else I can check to assure "long term health".

setup e-mail (Setup->Mail) and configure and boot-enable the "smart" and  "mdadm" Services->System, so you will be warned of current and possible future  errors.

You might also fine-tune the fan speed (Services->System->sysctrl) for your actual box/disk/ambient temperatures.

[ObviousMan]

On Friday, May 6, 2016 at 10:10:09 AM UTC-5, João Cardoso wrote:

I just want to make sure there are no elements of Alt-F, etc. are missing,

The Alt-F folder only contain files from new or updated Alt-F packages (Packages->Alt->F), so if you don't need any that folder is not needed and might even cause some annoyances. *Always* use the Packages webui to manipulate the folder, or the box might froze -- read its README.txt.

 

so I wanted to ask what else I can check to assure "long term health".

setup e-mail (Setup->Mail) and configure and boot-enable the "smart" and  "mdadm" Services->System, so you will be warned of current and possible future  errors.

You might also fine-tune the fan speed (Services->System->sysctrl) for your actual box/disk/ambient temperatures.

Thank you sir.