Which volume/filesystem for a 15.0TB RAID hard drive?
Michele
I have a server with hardware RAID, so in Linux I see only a giant hard
drive which has a capacity of 15.0TB (specifically, /dev/sda1)
My idea is to have a single partition on this disk for the easiness and
simplicity of management (I want to use all this space for a file server).
However, I know that with a single partition, a fsck light will take
days to complete.
In addition, I would like to have a scalable and flexible solution: for
example, in the future, I would like to add a cluster and unify all the
hard drive space in one giant hard disk, in order to have a transparent
underlying hardware.
So I was thinking about some solutions to counter the problem and
satisfy my needs, like LVM, LVM+GPT and GFS2.
What do you think about?
What do you suggest me?
Thank you.
Michele
Nathan Keel
>
> My idea is to have a single partition on this disk for the easiness
> and simplicity of management (I want to use all this space for a file
> server).
A single, giant partition is not a good idea for good management. Do
you mean you'll have partitioned out the normal directories and just
have an extra big one for storage, or to install the OS on?
Greg Russell
news:h7hfgf$3qt$1...@daisy.noc.ucla.edu...
> I have a server with hardware RAID, so in Linux I see only a giant hard
> drive which has a capacity of 15.0TB (specifically, /dev/sda1)
>
> My idea is to have a single partition on this disk for the easiness and
> simplicity of management (I want to use all this space for a file server).
Is your residential house just a box, with no bedrooms, no kitchen, no
toilet, no closet storage, no garage? Just a big box where everything goes
in it?
Your use of the term "management" is a contradiction.
Rahul
news:7g3325F...@mid.individual.net:
> Is your residential house just a box, with no bedrooms, no kitchen, no
> toilet, no closet storage, no garage? Just a big box where everything
> goes in it?
>
> Your use of the term "management" is a contradiction.
>
Is this such a bad idea? Assuming it is *just* for storage and not OS /
swap etc?
What if have 500 equvalent users and I need to farm out /home space.
Conceptually a single large RAIDed partition is what I'd have done in my
naiive understanding. Is it because of implimentation reasons that you'd
recommend otherwise?
--
Rahul
David W. Hodgins
> Is this such a bad idea? Assuming it is *just* for storage and not OS /
> swap etc?
It can make problems for running backups or fsck. What type of data will
be stored there, in terms of expected initial volume, rate of growth,
frequency of updates, uptime requirements, etc.
Depending on the actual data needs, I'd probably use a single lvm physical
volume, with logical volumes for the data, and a snapshot. Also depends
on what other storage you have available, and how backups will be done.
As to the actual filesystem within the logical volume, I'd go with ext4,
as it will easily handle that size. See
http://en.wikipedia.org/wiki/Comparison_of_file_systems
for other options that can handle the size.
Regards, Dave Hodgins
--
Change nomail.afraid.org to ody.ca to reply by email.
(nomail.afraid.org has been set up specifically for
use in usenet. Feel free to use it yourself.)
Michele
> On Mon, 31 Aug 2009 20:33:09 -0400, Rahul <nos...@nospam.invalid> wrote:
>
>> Is this such a bad idea? Assuming it is *just* for storage and not OS /
>> swap etc?
>
> It can make problems for running backups or fsck. What type of data will
> be stored there, in terms of expected initial volume, rate of growth,
> frequency of updates, uptime requirements, etc.
>
> Depending on the actual data needs, I'd probably use a single lvm physical
> volume, with logical volumes for the data, and a snapshot. Also depends
> on what other storage you have available, and how backups will be done.
>
> As to the actual filesystem within the logical volume, I'd go with ext4,
> as it will easily handle that size. See
> http://en.wikipedia.org/wiki/Comparison_of_file_systems
> for other options that can handle the size.
>
> Regards, Dave Hodgins
>
How long does a fsck will last on 1TB partition?
Does ext3cow works inside a LVM physical volume?
Thanks
Sidney Lambe
> Michele wrote:
>
>>
>> My idea is to have a single partition on this disk for the easiness
>> and simplicity of management (I want to use all this space for a file
>> server).
>
> A single, giant partition is not a good idea for good management.
The ignorant are always so sure of themselves. That's how they stay
ignorant.
It's a tossup on multiple partitions vs. a single partition.
There's some protection in the multiple choice, but nothing more
frustrating that running into "no space left on this device" when
you are in the middle of something. If it's an important cron job
it can really screw things up.
> Do you mean you'll have partitioned out the normal directories
> and just have an extra big one for storage, or to install the
> OS on?
NOW he asks that question...
And right after telling him that he had no choices because a
single giant partition was a bad idea.
Perhaps this will help people understand why I don't read his
replies to my posts or any replies to them.
Sid
--
Shell Interface Mailing List
screening address: usenet4444
[AT] gmail (dot) com
David W. Hodgins
> How long does a fsck will last on 1TB partition?
I only have experience with one recent case. I set up a system last
week with two 7200rpm 1.5TB drives. I setup one physical volume
on each drive, and the /home /and /mnt/backup filesystems as 1.2TB
ext4 filesystems in logical volumes with 8kb stripes, so the files
and filesystems are spread over the two drives.
With an initial 15GB of actual data on the /home, and 4GB (compressed
copy of /home) on /mnt/backup, each of those took about 3 minutes
to fsck, using a quad core cpu with 4GB of ram. I didn't bother
using a snapshot in this case, as the system is not in regular use
at night.
Based on that, I'd expect fsck on a 15TB filesystem to be in the 30-45
minute range. Maybe someone else here will have experience with
that size of filesystem.
> Does ext3cow works inside a LVM physical volume?
As long as the kernel can handle both, there shouldn't be any problem.
I haven't used ext3cow, but given the way lvm works, I'd be very
surprised if it caused any problems. I have used ext2/3/4, reiserfs,
xfs, vfat, and probably a few others that I've experimented with.
Nathan Keel
> Nathan Keel <na...@gm.ml> wrote:
>> Michele wrote:
>>
>>>
>>> My idea is to have a single partition on this disk for the easiness
>>> and simplicity of management (I want to use all this space for a
>>> file server).
>>
>> A single, giant partition is not a good idea for good management.
>
> The ignorant are always so sure of themselves. That's how they stay
> ignorant.
Hey dipshit, you said you killfiled me and ignore my posts because I'm
this "Dan C" guy posting as an alias (along with thousands of other
people, according to you), so why are you replying? Make up your mind
in what you want to claim.
Secondly, you think it's ignorant to partition a disk for an OS install?
You are seriously dumb, which is evidenced by pretty much every single
claim you've ever made on usenet. I simply suggested a giant single
partition isn't as wise, because it's not, if it was for an OS install.
Things are more efficient, safer, and more secure with non single
partitions, but you don't understand how to use a GDE or the CLI, so of
course you'd not know those basics about partitioning schemes and why
one is a better idea. It is not good management to have a single,
giant partition.
> It's a tossup on multiple partitions vs. a single partition.
No, it's not.
> There's some protection in the multiple choice, but nothing more
> frustrating that running into "no space left on this device" when
> you are in the middle of something. If it's an important cron job
> it can really screw things up.
There is a large difference, and if you think that running out of space
is the primary concern with all of the options available for preventing
that, then you just go on to show your ignorance.
>> Do you mean you'll have partitioned out the normal directories
>> and just have an extra big one for storage, or to install the
>> OS on?
>
> NOW he asks that question...
That's right, I asked them if it was for an OS install or just for
storage (big difference). I qualified the prior statement. Get a
grip.
> And right after telling him that he had no choices because a
> single giant partition was a bad idea.
You're adding things I never said. I never said there were no other
choices, I simply qualified the prior statement.
> Perhaps this will help people understand why I don't read his
> replies to my posts or any replies to them.
>
> Sid
>
You CLAIM you don't read my posts, but you certainly (always) do. You
are so wound up and gung ho on trying to berate people that point out
your ignorance, that you yet again go nuts in a reply and try and look
smart, but you've only proven you lack basic reading skills.
Alister
if it is raid 0 (so that multiple drives are being added to each other)
I would seriously think again one dead drive will result in a major loss
of data
In my opinion( & i am now prepared to be shot down in flames) it would
be better to go for a raid 5 option to increase reliability at the
expense of storage space, do you really need 15 TB of storage?
It would help to know exactly what mix of drives you have.
David Brown
I'll bite on that one (though I expect a counter-attack)...
Raid 5 would be a bad idea here, although not as bad as raid 0. To get
15 TB, there are a lot of large disks involved - say 10 1.5 TB disks if
it is raid 0. The trouble with raid 5 in a setup like this is that if
one disk fails, there's a pretty good chance of another disk failing
while you replace the bad one and re-build.
Statistically speaking you have a much higher chance of two disks
failing in such a setup than if you look at the MTBF ratings for the
disks independently, since they are typically the same type of drive
from the same batch, and are run under the same operating conditions. A
raid 5 rebuild is about as much stress as you can give the disks, so a
second failure during rebuild is too likely for comfort. And if you
read some of the online doomsday reports about undetected read errors,
you can practically guarantee failure of the whole raid during a rebuild.
The traditional fix is to use raid 6, which gives you two redundant
parity drives - it's certainly better than raid 5.
A better choice, if you can afford the extra disks, is raid 1+0 (a
stripe of mirrors). Rebuild is much easier and faster, and safer, and
you have better performance than raid 5 for many tasks (especially if
you are using Linux software raid 10, though that's not the case here).
David Brown
I would start off with LVM on the drive. That lets you add (and remove)
partitions as you feel the need. Pick file systems that grow easily
(xfs, ext3+) online so that you can expand the partitions as you need.
That way you don't have to make all your decisions up front.
For fsck, be careful of two things - running times, and memory usage.
xfs fsck takes memory proportional to the size of the file system
partition, which could be a real problem with a huge partition. Newer
filesystems such as ext4 and btrfs (if you like to play with cool new
toys) are very much faster than ext3- at fsck since they only need to
check used areas and inodes.
A nice trick with LVM and fsck is to take a temporary snapshot of the
running partition and do an fsck on the snapshot. If this is okay, then
you know your real partition is okay (at least it was at the time of the
snapshot). If it fails, it's time to take the partition offline and run
a real fsck on it. But for the normal case of clean fsck as a check,
you can run it in the background without affecting the use of the main
partition(s) (other than losing a little IO throughput).
Finally, if you've got 15 TB of raid data, make sure you have a very
good backup procedure thought out, as well as having spare parts on-hand
for the raid controller and a couple of extra disks (preferably hot spares).
terryc
> Hi everybody,
> I have a server with hardware RAID, so in Linux I see only a giant hard
> drive which has a capacity of 15.0TB (specifically, /dev/sda1)
what sort of data is being served?
file size, average read/write, etc.
how are you backing it up?
--
Great advances in Debian Linux; post a bug report and get spam in three
days.
Alister
I have never used raid & was not familiar with raid 6.
My main point was to find out what setup the op is actually using before
giving advice on any partition structure.
philo
<snip>
Do not feed the troll
David Brown
I'll admit to being more experienced in raid theory than raid practice
(I've researched it a fair bit recently, and thought about how things
work - especially when they don't work. But I've only tried out raid 0,
raid 1 and raid 10 in practice, having only had two disks on my test
machine).
Raid 6 is like raid 5 only more so. It gives better protection since it
can survive two disk failures, but it has more calculation overhead and
the same "raid 5 hole" if you write less than a stripe at a time.
> My main point was to find out what setup the op is actually using before
> giving advice on any partition structure.
>
And that is very sound advice. It's also important to know what he
wants to do with the system (different setups have different balances
between performance, reliability and disk space).