setting up JBOD and 3TB disks

[Andy O.]

I have 2Tb and 3Tb drives and i have hard time setting up JBOD or RAID for that matter, since RAID utility does not recognize the big partition. Is there is a command line that can setup raids. I'm not a linux person and any help would be much appreciated.

Both drives are formatted ex4 and recognized at their full capacity.

RAID Creation

fdisk: device has more than 2^32 sectors, can't use all of them

Dev.	Type	Comp 1	Comp 2	Spare
md0	JBD Raid 0 Raid 1 Raid 5	none	none	none

Thank you

[Joao Cardoso]

As the RAID online help says: 

  In Alt-F, for a physical partition to be allowed to be a RAID component its partition type must be RAID.

This is the reason why you can't select any component, as there are none.

To be accurate, the partitions that appear under "Comp" are either partitions of an already existing RAID array (such as one build using the D-Link firmware), or partitions of type RAID.

So, you have to change the partition type of your partition to type RAID, which can be done using the Disk Partitioner: select the disk which you want to partition in the upper section, than uncheck the 'Keep' checkbox in the partition which you want to change, change  the partition type to type RAID, *DON'T* change anything else (changing the partition size and undoing, or hitting the Advance button and returning to Basic *IS* changing), then hit the Partition button.

As the Disk Partitioner help page says, near the end:

   NOTES:
   -To change only the type of one partition, un-check its Keep check-button, select the new partition type and hit the Partition button.

This will work fine for the 2TB disk, and without loosing your data (if you follow the procedure exactly, without any variations)

After following this procedure the partition will appear as one RAID candidate component in the RAID web page.

*** BUT ***

You say:

   1-I have 2Tb and 3Tb drives

Is your 3TB disk GPT partitioned? Have you followed the command-line procedure outlined under the "DNS-323, current Alt-F firmware, and 3TB drives " topic?

If you have, you have to change '--typecode=2:8300' to '--typecode=2:fd00' where relevant (it's the partition type)

To not redo the whole procedure again, just issue the command

   sgdisk --typecode=2:fd00 /dev/sdX

where the '2' is the second partition, 'fd00' is the RAID partition type, and  X is either 'a' or 'b', your 3TB GPT partitioned disk.

   2-Both drives are formatted ex4 and recognized at their full capacity.

Can you loose all your data? I'm almost sure that creating a JBOD will make all your data inaccessible. At least the data contained in the second component partition will.

Please report back your results

[Andy O.]

Thank you for quick response. Yes i did setup 3gb as outlined in  "DNS-323, current Alt-F firmware, and 3TB drives " topic. I will try it in the evening and report back. Both drives are empty so i can do anything and everything needed. 

Thank you again.

Andy

[Andy O.]

Great, did as noted:

1) sgdisk --typecode=2:fd00 /dev/sdb (in my case)

2) hanged partition to RAID

3)Rebooted, 

4)when setting up JBOD the system asked to un-mount 3tb hdd>unmounted and done!

VOILA,  md0 is 55% done

Thank you, would not done it without your help (reading sgdisk man drove me crazy)

-Andy

On Friday, May 18, 2012 10:12:17 AM UTC-5, Joao Cardoso wrote:

[Joao Cardoso]

On Saturday, May 19, 2012 3:19:38 AM UTC+1, Andy O. wrote:

Great, did as noted:

1) sgdisk --typecode=2:fd00 /dev/sdb (in my case)

2) hanged partition to RAID

3)Rebooted, 

4)when setting up JBOD the system asked to un-mount 3tb hdd>unmounted and done!

VOILA,  md0 is 55% done

Good it worked.

But you didn't need to reboot, this is not windows ;-)

With a few exceptions, in linux you don't need to reboot when you install or remove sofware, install, partition or format disks, etc.

[Andy O.]

RAIDDev. Capacity Level State Status Action Done ETA
md0 4656.5 GB linear clean


Mounted FilesystemsDev. Label Capacity Available FS Mode Dirty Automatic FSCK in

md0 1.8TB 0B ext4 RW 33 mounts or 172 days

Looks like ran into the 1st problem. 2gb hdd got filled up, since it was 1st in the chain and both FTP and windows (samba)say no space. Anybody has any suggestions?

Thanks

Andy O.

[Joao Cardoso]

So raid says that mdo has 4tb cap. but filesystem says mdo only has 1.8tb cap.

Are you sure that the filesystem was not created on one of the raid components before the raid creation itself? The oposite should be done, create first the raid device (mdo) then the filesystem. That would explain the symptom.

The classical way of solving the problem is to enlarge the filesystem so as to use all the device space, use Disk->Filesystem->enlarge.
But I don't know exactly how/what happened, and you have data on it.
Read the filesystem online help page.

On May 26, 2012 7:42 PM, "Andy O." <andr...@gmail.com> wrote:

RAIDDev. Capacity Level State Status Action Done ETA
md0 4656.5 GB linear clean


Mounted FilesystemsDev. Label Capacity Available FS Mode Dirty Automatic FSCK in

md0 1.8TB 0B ext4 RW 33 mounts or 172 days

Looks like ran into the 1st problem. 2gb hdd got filled up, since it was 1st in the chain and both FTP and windows (samba)say no space. Anybody has any suggestions?

Thanks

Andy O.

On Saturday, May 19, 2012 10:19:38 AM UTC-5, Joao Cardoso wrote: > > > > On Saturday, May 19, 201...

To view this discussion on the web visit https://groups.google.com/d/msg/alt-f/-/jIjrMJNuDvwJ.

To post to this group, send email to al...@googlegroups.com. To unsubscribe from this group, send e...

[Joao Cardoso]

I can devise some diagnostics to find what really happened, but you gave to wait till late tomorrow.

On May 26, 2012 9:23 PM, "Joao Cardoso" <joao.fs...@gmail.com> wrote:

So raid says that mdo has 4tb cap. but filesystem says mdo only has 1.8tb cap.

Are you sure that the filesystem was not created on one of the raid components before the raid creation itself? The oposite should be done, create first the raid device (mdo) then the filesystem. That would explain the symptom.

The classical way of solving the problem is to enlarge the filesystem so as to use all the device space, use Disk->Filesystem->enlarge.
But I don't know exactly how/what happened, and you have data on it.
Read the filesystem online help page.

> > On May 26, 2012 7:42 PM, "Andy O." <andr...@gmail.com> wrote: > > RAIDDev. Capacity Level Sta...

[Andy O.]

looks like expanding the volume did the trick.

Thank you for your advise.

-Andy O.

[ctiger]

Andy/Joao,

Could you guys list a summary steps on how to setup two drives as a JBOD? I have a 1TB drive and 3TB drive, My dns323 is using only the 1TB drive right now.

Thanks,
Raymond

[Joao Cardoso]

On Wednesday, July 18, 2012 8:05:31 AM UTC+1, ctiger wrote:

Andy/Joao,

Could you guys list a summary steps on how to setup two drives as a JBOD? I have a 1TB drive and 3TB drive, My dns323 is using only the 1TB drive right now.

It depends if you have data that you can't backup on your current 1TB drive and how it is setup -- as a "standard", non RAID disk?

If you can backup your data first, then you can follow Mike's instructions posted under the "DNS-323, current Alt-F firmware, and 3TB drives" forum topic: https://www.mikekirk.org/index.php/2012/06/24/using-3tb-drives-in-your-d-link-dns-323/ 

Just replace at the RAID creation step "–level=raid1" with "–level=linear".

I would also advise you to use the second version of the 'mdadm --create' command, the one with "–metadata=0.90 " to create a D-Link compatible RAID.

Read also Will Kallender post regarding copying/pasting of Mike's instructions.

The fact that you have one 1TB and one 3TB disks and not 2x3TB disks is irrelevant.

 

[ctiger]

Got it, I'll give it a try tonight. My 1TB drive is empty since I just setup the dns323 yesterday and flashed it with alt-f.

Thanks a lot for making such a great firmware, the mounted windows shares(public(RO), public(RW) and Users) make so much sense!

Raymond

On Wednesday, July 18, 2012 6:17:20 AM UTC-7, Joao Cardoso wrote:

On Wednesday, July 18, 2012 8:05:31 AM UTC+1, ctiger wrote:

Andy/Joao,

Could you guys list a summary steps on how to setup two drives as a JBOD? I have a 1TB drive and 3TB drive, My dns323 is using only the 1TB drive right now.

[ctiger]

I tried following the steps but failed at second step at both disk(sda 1tb, sdb3tb):

login as: root
ro...@10.0.0.100's password:
COLUMNS=80;LINES=24;export COLUMNS LINES;
# sgdisk -zap-all /dev/sda
# sgdisk -set-alignment=8 -new=1:64:+512M -typecode=1:8200 /dev/sda

***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format.
***************************************************************

Could not change partition 5698's type code to !
Could not create partition 5658 from 64 to 1048639
Non-GPT disk; not saving changes. Use -g to override.
Error encountered; not saving changes.
# sgdisk -zap-all /dev/sdb
# sgdisk -set-alignment=8 -new=1:64:+512M -typecode=1:8200 /dev/sdb
Could not change partition 5698's type code to !
Could not create partition 5658 from 64 to 1048639
Error encountered; not saving changes.
#

Am I missing anything?

Thanks,

Raymond

On Wednesday, July 18, 2012 6:17:20 AM UTC-7, Joao Cardoso wrote:

On Wednesday, July 18, 2012 8:05:31 AM UTC+1, ctiger wrote:

Andy/Joao,

Could you guys list a summary steps on how to setup two drives as a JBOD? I have a 1TB drive and 3TB drive, My dns323 is using only the 1TB drive right now.

[ctiger]

Just figured that I need to replace the long dash with two ascii "--' and it works now. For the mdadm command, I had to remove the "--bitmap=internal" since it complained about it's not meaningful for --level=linear.

I started to transfer files to the new volume but the transfer rate is only less than 3MB/s, any idea why it's so slow? I think it's 20MB/s before when I used the 1.09 firmware.

Raymond

[Joao Cardoso]

On Friday, July 20, 2012 8:55:57 AM UTC+1, ctiger wrote:

Just figured that I need to replace the long dash with two ascii "--' and it works now.

Yes, as Will Kallender post under the "DNS-323, current Alt-F firmware, and 3TB drives" topic, regarding copying/pasting of Mike's instructions, says.

 

For the mdadm command, I had to remove the "--bitmap=internal" since it complained about it's not meaningful for --level=linear.

 

Right..

 

I started to transfer files to the new volume but the transfer rate is only less than 3MB/s, any idea why it's so slow?

Using samba? No idea, shouldn't be that low.

Are you on a GB network? What does the Status Page says in the Network section?

What filesystem do you have? D-Link uses ext2, which is faster then the more recent ext3/ext4...

Do you also had linear RAID under DLink firmware? Each layer needs more CPU resources...

 

I think it's 20MB/s before when I used the 1.09 firmware.

I never got that figures with DLink firmware.

Most posted results on this subject are biased, most of the time because of caching effects on the desktop computer, where the huge amount of memory favors caching; also small files transfers biases results.

When testing, I always transfer a single multi-GB file, *once*, and measure *sustained* transfer speed, testing both reads and writes (using different files, once)

The measured file transfer speed involves network transfer throughput and  filesystem throughput.

But 3MB is a low figure, I get between 8 and 12MBps using samba.

[ctiger]

I am using samba and backing up stuff from windows.  Yes, it was ext2 and now I am using ext4. The network status right now is like:

Speed: 1000Mb/s MTU: 4088 TX: 1.0 GiB Rx: 1.1 GiB

After changed the frame size to 4088, I am getting 5.6MB/s for writing but still slow. I think what I got for writing before was around 15MB/s.

Raymond

[Joao Cardoso]

On Friday, July 20, 2012 4:28:11 PM UTC+1, ctiger wrote:

I am using samba and backing up stuff from windows.  Yes, it was ext2 and now I am using ext4. The network status right now is like:

Speed: 1000Mb/s MTU: 4088 TX: 1.0 GiB Rx: 1.1 GiB

After changed the frame size to 4088, I am getting 5.6MB/s for writing but still slow. I think what I got for writing before was around 15MB/s.

Raymond

Using ext3 on a RAID1 on a 100Mbps network with a MTU of 1500 I get the following from my linux laptop when reading/writing a 1.5GB file:

ftp get/put: 11.2/9.5MBps

smbclient get/put 9.0/8.1MBps

But in the same conditions, only using 'smbget' instead of 'smbclient', I only got 5.4MBps!

So I'm not surprised that your figures with MS are different.

I wasn't able to change the (too old) laptop MTU, but I remember getting around 47MBps (or was it 37MBps?) with ftp and a MTU of around 8K on a Gbps network

Another reason to have lower throughput values than with D-Link firmware is that Alt-F uses samba 3.5.x, which is much more bloated than D-Link 3.0.x.

Of course desktop users with fast CPUs and lots of memory will not notice that newer software versions use more resources than older versions, but on a limited CPU/memory box it will be noticed.

I have seriously considered downgrading Alt-F samba to another, lower version, but I'm missing the motivation.

:-(

[ctiger]

Do you think switching to ext2 would make a big difference?

[Joao Cardoso]

On Saturday, July 21, 2012 1:59:38 AM UTC+1, ctiger wrote:

Do you think switching to ext2 would make a big difference?

Judge for yourself, read the top-posted "Filesystem and disk layout performance on 0.1B4". The conclusions should still apply.

But notice that my figures show a 9MBps read and 8MBps write throughput on a 100Mbps network with a 1500 MTU! The triple of yours 3MBps.

Have you already rebooted you MS machine? It does marvels on my wife Vista PC :-o

[ctiger]

Yes, I rebooted both the windows and dns323 and it did made a difference, it's at around 8-13MB/s for writes. I said I was getting 15MB/s for writes before switching to alt-f and I'll have to take it back because it's the speed for writing to my sata dock station not dns323.

Thanks again for all the helps!
Raymond

[junk...@gmail.com]

Jodo,

I am rebuilding my dns-323 and the mdadm command doesn't work with this message:

# mdadm --create /dev/md0 --metadata=0.90 --chunk=64 --level=linear --raid-devic
es=2 /dev/sda2 /dev/sdb2
mdadm: super0.90 cannot open /dev/sda2: Device or resource busy
mdadm: /dev/sda2 is not suitable for this array.
mdadm: create aborted

Any idea?

[junk...@gmail.com]

It works now after I run command:
mdadm --manage /dev/md0 --stop