Problem creating 6TB partition...

[Stefan Drees]

Hi,
yesterday i installed debian etch amd64 on an FSC Server with an dell
md1000 diskarray attached.
The total size of the diskarray is 6TB and i can see the complete 6TB in
fdisk and cfdisk.
If i try to create an 6TB partition, it seems to work but after leavinf
fdisk or cfdisk
and reentering it shows me only an 1,5TB partition and i can´create
another partition.

Any hints? What can i do?

Best regards
Stefan D.

--
To UNSUBSCRIBE, email to debian-amd...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

[Roberto C. Sánchez]

On Sun, Apr 29, 2007 at 02:21:37PM +0200, Stefan Drees wrote:
> Hi,
> yesterday i installed debian etch amd64 on an FSC Server with an dell
> md1000 diskarray attached.
> The total size of the diskarray is 6TB and i can see the complete 6TB in
> fdisk and cfdisk.
> If i try to create an 6TB partition, it seems to work but after leavinf
> fdisk or cfdisk
> and reentering it shows me only an 1,5TB partition and i can´create
> another partition.
>
> Any hints? What can i do?
>

Some RAID hardware has a physical partition size (or logical disk size,
in terms of what Intel calls it) limitation. At work we have an HP
StorageWorks tray with ~5.6TB of SCSI disks in it. The Intel RAID card
which controls does not allow partitions bigger than 2TB. The solution
for us was to create 3 ~1.8TB partitions and put them together as 3
physical volumes in a LVM volume group. We then just created one bug
logical volume of ~5.6TB.

Regards,

-Roberto

--
Roberto C. Sánchez
http://people.connexer.com/~roberto
http://www.connexer.com

[Stefan Drees]

Thanks, how can i find out about the dell md1000?
The technical data doesn´t say anything about a limit.

Roberto C. Sánchez schrieb:

--

[Gabor Gombas]

On Sun, Apr 29, 2007 at 02:21:37PM +0200, Stefan Drees wrote:

> Any hints? What can i do?

AFAIK With a DOS-compatible partition table you'll not be able to create
a partition bigger than 2TB. Forget the *fdisk variants, use parted, and
use GPT instead of DOS partitions.

Gabor

--
---------------------------------------------------------
MTA SZTAKI Computer and Automation Research Institute
Hungarian Academy of Sciences
---------------------------------------------------------

[Stefan Drees]

Ok, so i can create one big volume with parted (gpt) or 3 <2TB
volume's (DOS) and make one big volume with lvm.
What´s the best way and which filesystem for such a big volume?

Last time i used EXT3 for an 2TB volume, everythings fine but
i need to disable the filesystem checks with tune2fs, because it
needs too much time for an check :-). I´m not felling really good
about that, is there a better solution?

Thanks.

Gabor Gombas schrieb:

> On Sun, Apr 29, 2007 at 02:21:37PM +0200, Stefan Drees wrote:
>
>
>> Any hints? What can i do?
>>
>
> AFAIK With a DOS-compatible partition table you'll not be able to create
> a partition bigger than 2TB. Forget the *fdisk variants, use parted, and
> use GPT instead of DOS partitions.
>
> Gabor
>
>

--

[Roberto C. Sánchez]

On Sun, Apr 29, 2007 at 07:00:46PM +0200, Stefan Drees wrote:
> Ok, so i can create one big volume with parted (gpt) or 3 <2TB
> volume's (DOS) and make one big volume with lvm.
> What´s the best way and which filesystem for such a big volume?
>
> Last time i used EXT3 for an 2TB volume, everythings fine but
> i need to disable the filesystem checks with tune2fs, because it
> needs too much time for an check :-). I´m not felling really good
> about that, is there a better solution?
>

There was a very long thread about what filesystem is best for large
partitions a while back on debian-user. It is in the archives if you
are interested.

Personally, I would use XFS (first choice) or JFS (second choice) if
those are available to you. Those are both available stock (as in no
need to recompile the kernel, assuming you use a Debian shipped kernel)
since Sarge. If you cannot use either of those, then ext3 is
acceptable. However, read the man page carefully and look at all the
options. You can choose some of the paramters at filesystem creation
time to minimize things like the amount of time it takes to perform a
fsck. For example, you can make your chunk sizes (or is it block
sizes?) bigger and have fewer superblock replicas, which will reduce the
time it takes to both create and fsck the filesystem. Of course, you
need to understand how the filesystem will be accessed in order to make
the best choices. That is, if you will have many small files, you don't
want to make the block size too big since it will waste much space. If
you will have mostly large files, then make the block sizes really big,
since you won't waste too much space but will make filesystem access
faster.

[Mike Dresser]

> Last time i used EXT3 for an 2TB volume, everythings fine but
> i need to disable the filesystem checks with tune2fs, because it
> needs too much time for an check :-). I´m not felling really good
> about that, is there a better solution?

Keep in mind if you go with XFS, you're going to need 10-15 gig of
memory or swap space to fsck 6tb.. it needs about 9 gig to xfs_check, and
3 gig to xfs_repair a 4tb array on one of my systems.. oh, and a couple
days to do either. :)

Mike

[Roberto C. Sánchez]

Is it an exponential growth in the amount of time it takes? I've had
some XFS partitions that were several hundred GBs (but not close to 1TB)
and those seemed to pass the fsck stage very quickly.

[Mike Dresser]

On Sun, 29 Apr 2007, Roberto C. Sánchez wrote:

> Is it an exponential growth in the amount of time it takes? I've had
> some XFS partitions that were several hundred GBs (but not close to 1TB)
> and those seemed to pass the fsck stage very quickly.

If i remember right, it's 1 gb of memory per TB of space, plus additional
memory overhead for X number of inodes.. i have both a large filesystem
and millions of hardlinks, so lots of inodes.

Mike

[Bernd Petrovitsch]

On Sun, 2007-04-29 at 19:00 +0200, Stefan Drees wrote:
[...]

> Last time i used EXT3 for an 2TB volume, everythings fine but
> i need to disable the filesystem checks with tune2fs, because it

I do this on all ext3 partitions. Replaying the journal should be enough
after a reboot if nothing especially bad happened (and I consider a
unclean shutdown not "especially bad", just "bad").
And journalling filesysems are here to exactly fix that problem.

> needs too much time for an check :-). I´m not felling really good
> about that, is there a better solution?

Generally, IMHO no. A fsck will cost a lot of time with all filesystems.

Bernd
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services

[Roberto C. Sánchez]

Interesting. The filesystem to which I was referring stays mostly
empty. That may explain the results I am seeing.

[Mike Dresser]

On Sun, 29 Apr 2007, Bernd Petrovitsch wrote:

> Generally, IMHO no. A fsck will cost a lot of time with all filesystems.

Some worse than others though.. looks like this 4tb is going to take 3
weeks.. it took about 3-4 hours on ext3.. If i had a couple gig of ram to
put in the server that'd probably help though, as it's constantly
swapping out a few meg a second.

Mike

[Stefan Drees]

Hello,
many thanks for all your tipps. I think i will use EXT3
and read a little bit about the man page to optimize.

Best regards.
Stefan D.

[Daniel Schröter]

Mike Dresser wrote:
>> Last time i used EXT3 for an 2TB volume, everythings fine but
>> i need to disable the filesystem checks with tune2fs, because it

>> needs too much time for an check :-). I´m not felling really good
>> about that, is there a better solution?
>

> Keep in mind if you go with XFS, you're going to need 10-15 gig of
> memory or swap space to fsck 6tb.. it needs about 9 gig to xfs_check, and
> 3 gig to xfs_repair a 4tb array on one of my systems.. oh, and a couple
> days to do either. :)

ext3 has the same "problem":
http://ukai.org/b/log/debian/snapshot/fsck_completed_but-2005-09-04-15-00.html

[Mike Dresser]

On Tue, 1 May 2007, Daniel Schröter wrote:

> ext3 has the same "problem":
> http://ukai.org/b/log/debian/snapshot/fsck_completed_but-2005-09-04-15-00.html

Sounds atypical to me, as it used to take around an hour or two to fsck
the same 4 TB partition before I changed over to XFS. It's been a couple
days now for the XFS, and probably another two weeks left. I've ordered
another 4 GB of memory for the system, so it'll have 5 GB to work with
soon.

Mike

[Alex Samad]

if its not too rude to ask, what do you guys have on these large partitions ?

Alex

[Mike Dresser]

On Fri, 4 May 2007, Alex Samad wrote:

> if its not too rude to ask, what do you guys have on these large partitions ?
>
> Alex
>

This one is the data store for backuppc.. backing up a bunch of
workstations and servers.

Mike

[Alex Samad]

On Fri, May 04, 2007 at 03:17:00PM -0400, Mike Dresser wrote:
> On Fri, 4 May 2007, Alex Samad wrote:
>
> >if its not too rude to ask, what do you guys have on these large
> >partitions ?
> >
> >Alex
> >
> This one is the data store for backuppc.. backing up a bunch of
> workstations and servers.

any advantage to splitting it up into say 3 x 2 and splitting the load between
3 partitions

seems a multi day fsck is a bit price to pay ?

>
> Mike
>
>

[Mike Dresser]

On Sat, 5 May 2007, Alex Samad wrote:

> any advantage to splitting it up into say 3 x 2 and splitting the load between
> 3 partitions
>
> seems a multi day fsck is a bit price to pay ?

the software doesn't support doing that though