[ale] Looking for recommendations on LVM + soft Raid on home server
Neal Rhodes
This time the re-install is to get the drive partitioning and soft RAID right. I didn't have the 2nd drive for the 2nd install.
Normally our prior Fedora servers have been
/dev/md1 on / type ext3 (rw)This time around I was thinking on using LVM, I guess to just get more experience with LVM. However, since you wouldn't want to risk resizing /boot or root filesystem, I see no point in them being in LVM.
/dev/md0 on /boot type ext3 (rw)
/dev/md2 on /u type ext3 (rw)
Primary drive is 1.5TB, of which 220GB is occupied by Windows7 boot, which I'd prefer to not disturb.
2nd drive is 1TB.
So, I'm thinking of a layout like this:
/dev/md0 on /boot type ext3 (rw) (whatever boot takes)
/dev/md1 on / type ext3 (rw) (about 50GB)
/dev/md2 on VolumeGroup00 (about 1TB)
And logical volumes for /home and /u, which can be resized as needed between /home and /u
/dev/sda? on /u2 (remaining 300GB, not Raid 1, just on the one bigger drive)
Is that going to work? Other thoughts?
Neal Rhodes
MNOP Ltd
Jim Kinney
A typical Fedora/RHEL/CEntOS install will put /boot on a partition and a single partition for a physical volume. That partition is split between / and swap so that swap can float as well if you add more RAM later. / is all one big partition typically but later version of desktop installs make a separate /home.
As you are looking at RAID as well, make the md devices and put LVM on top. The caveat is you want swap on non-RAID as it handles itself sort of raid-like already. (That said, swap on raid is reliable but it gets slow on heavy used systems)
md0 for /boot
sda2 for swap
sdb2 for swap
md1 for LVM pv_main
subdivide pv_main for OS layout. into /, /home, /var, etc...
_______________________________________________
Ale mailing list
A...@ale.org
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo
--
--
James P. Kinney III
As long as the general population is passive, apathetic, diverted to consumerism or hatred of the vulnerable, then the powerful can do as they please, and those who survive will be left to contemplate the outcome.
- 2011 Noam Chomsky
http://heretothereideas.blogspot.com/
Chuck Payne
Are you using a card to do your raid? If you are doing a soft raid,
leave /boot /dev/sda1, I haven't seen a box use /dev/mdx for boot.
Just like I haven't see a box use /dev/VolGroup/LogVolboot. Most
software raid have to be load, until the drivers are loaded, how will
your server know you are using a softraid.
If it's a hardware raid, then by all means set up the /boot as a read.
--
Terror PUP a.k.a
Chuck "PUP" Payne
(678) 636-9678
-----------------------------------------
Discover it! Enjoy it! Share it! openSUSE Linux.
-----------------------------------------
openSUSE -- en.opensuse.org/User:Terrorpup
openSUSE Ambassador/openSUSE Member
Community Manager -- Southeast Linux Foundation (SELF)
skype,twiiter,identica,friendfeed -- terrorpup
freenode(irc) --terrorpup/lupinstein
Register Linux Userid: 155363
Have you tried SUSE Studio? Need to create a Live CD, an app you want
to package and distribute , or create your own linux distro. Give SUSE
Studio a try. www.susestudio.com.
See you at Southeast Linux Fest, June 8-10, 2012 in Charlotte, NC.
www.southeastlinuxfest.org
Lightner, Jeff
Athena®, Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer
---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------
James Sumners
2) The advantage is that you can add space later. Or you can subtract
space if you really need to (like I had to do yesterday). Clearly this
comes at a cost of uptime. But that's what scheduled maintenance is
for.
Also, as Jim Kinney mentioned, I don't even bother with creating
separate partitions for /home, /var, etc. anymore. Disk space is so
cheap now that it doesn't make sense in most situations. So being able
to add to the / pool at a whim is real nice.
Actually, I sort of take 2 back. You can expand / while it is online,
you just can't shrink it (of course, how did you get the new drive in
there without down time? Does your home server support hot swapping?).
If a new drive is available then you can:
$ fdisk /dev/newdisk # create a partition on the new disk (e.g. /dev/sdb1)
$ pvcreate /dev/sdb1
$ vgextend /dev/LogVol00/root /dev/sdb1
$ lvresize -l +100%FREE -r /dev/LogVol00/root
On Tue, Mar 20, 2012 at 07:56, Neal Rhodes <ne...@mnopltd.com> wrote:
> How is it that you can resize / without booting to a rescue disk? And if
> you have to boot to a CD/Usb to resize root, well, is there any advantage to
> having / on LVM, or would it be safer to have / a regular non-lvm
> filesystem, so that the thing is more likely to survive a variety of events?
--
James Sumners
http://james.roomfullofmirrors.com/
"All governments suffer a recurring problem: Power attracts
pathological personalities. It is not that power corrupts but that it
is magnetic to the corruptible. Such people have a tendency to become
drunk on violence, a condition to which they are quickly addicted."
Missionaria Protectiva, Text QIV (decto)
CH:D 59
James Sumners
But I'm really just getting used to LVM myself.
Neal Rhodes
Now, I've already messed with logging in as root, unmounting /home and /u, and resizing both.
However, I had to unmount them in order for the native LVM to be willing to do that. which makes perfect sense.
How is it that you can resize / without booting to a rescue disk? And if you have to boot to a CD/Usb to resize root, well, is there any advantage to having / on LVM, or would it be safer to have / a regular non-lvm filesystem, so that the thing is more likely to survive a variety of events?
Lightner, Jeff
Users can fill up root by writing huge things to /tmp or /var/tmp, putting things in /usr/share or /usr/local (including logs). /var can fill up due to things like yum cache or logs in /var/logs. Keeping root (with /etc as part of root) separate from these other filesystems helps to keep from corrupting passwd and other files due to a full filesystem.
Oddly enough the reason why I make separate filesystems is the same you gave - disks are cheap so I don't need to try to avoid splitting things up in multiple LVs/filesystems in order to save space root might need someday.
-----Original Message-----
From: ale-b...@ale.org [mailto:ale-b...@ale.org] On Behalf Of James Sumners
Sent: Tuesday, March 20, 2012 8:30 AM
To: ne...@mnopltd.com
Cc: Atlanta Linux Enthusiasts
Subject: Re: [ale] Looking for recommendations on LVM + soft Raid on home server
Athena®, Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer
---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------
James Sumners
logic. Thankfully, I don't have to worry about very many people
directly connecting to the system. Most only connect to the services
provided by the system.
On Tue, Mar 20, 2012 at 09:02, Lightner, Jeff <JLig...@water.com> wrote:
> I disagree with NOT having separate /var, /usr, /home, /tmp etc...
> Users can fill up root by writing huge things to /tmp or /var/tmp, putting things in /usr/share or /usr/local (including logs). /var can fill up due to things like yum cache or logs in /var/logs. Keeping root (with /etc as part of root) separate from these other filesystems helps to keep from corrupting passwd and other files due to a full filesystem.
>
> Oddly enough the reason why I make separate filesystems is the same you gave - disks are cheap so I don't need to try to avoid splitting things up in multiple LVs/filesystems in order to save space root might need someday.
Lightner, Jeff
Why Oracle uses /tmp space rather than its own assigned space is another battle entirely so don't ask.
-----Original Message-----
From: ale-b...@ale.org [mailto:ale-b...@ale.org] On Behalf Of James Sumners
Sent: Tuesday, March 20, 2012 9:13 AM
To: Atlanta Linux Enthusiasts
Subject: Re: [ale] Looking for recommendations on LVM + soft Raid on home server
Athena®, Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer
---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------
Leam Hall
Morning Neil!
If you're looking to learn, then life becomes a lot more fun! Some
things to consider might be adding a swap space partition and separating
/var, /usr, and /home from root (which you have done partly). Swap is a
separate partition at the BIOS level though you can add swap volumes
later under LVM. However, I prefer a swap partition because if the
machine needs to swap then adding overhead for LVM seems against the grain.
With /home separate you can just tar it up for archive, move it off the
machine, and then restore it back once you rebuild. I've been doing this
for years and still have files from 10 years ago even though a few of my
machines have crashed.
Moving /var and /usr off root helps because they tend to grow a lot.
That lets you move things around easier though it's a bit trickier than
/home.
What are you using /u for?
Leam
James Sumners
On Tue, Mar 20, 2012 at 09:23, Lightner, Jeff <JLig...@water.com> wrote:
> Even with indirect connections you can have problems. For example I've seen a few occasions where badly structured Oracle queries essentially selected all rows and filled up /tmp space.
>
> Why Oracle uses /tmp space rather than its own assigned space is another battle entirely so don't ask.
--
Robert
I've usually broken these out too.. And was surprised to find that Fedora 16
spits out a warning if you have /usr on it's own partition. The seems to be
something to do with the new systemd, which manages the init scripts.
http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
Robert
>and /u, and resizing both.
>
>However, I had to unmount them in order for the native LVM to be willing
>to do that. which makes perfect sense.
I haven't found that to be the case. I regularly resize /, /home, /usr, etc on
my machines, with no problem. This is for ext* filesystems, I don't know if
other filesystems have more restrictions. It goes something like this:
# lvextend -L +4g /dev/vg/root
# resize2fs /dev/vg/root
It's a little more complicated for shrinking, and I think that you do need a
rescue disk to shrink anything you can't unmount, like /.
Lightner, Jeff
Interesting read. Not an issue I've run into in RHEL (including RHEL6) yet.
-----Original Message-----
From: Robert [mailto:r...@ale.spam.futz.org]
Sent: Tuesday, March 20, 2012 11:43 AM
To: Atlanta Linux Enthusiasts
Cc: Lightner, Jeff
Subject: partitioning and /usr [was Re: [ale] Looking for recommendations on LVM + soft Raid on home server]
>I disagree with NOT having separate /var, /usr, /home, /tmp etc...
I've usually broken these out too.. And was surprised to find that Fedora 16
spits out a warning if you have /usr on it's own partition. The seems to be
something to do with the new systemd, which manages the init scripts.
http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
Athena(r), Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer
---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------
Jim Kinney
Lightner, Jeff
/usr because I’ve been doing it for years on other platforms as the UNIX System V way of doing things. I even create /opt although other than Dell OpenManage I’ve not found many Linux apps that use /opt.
However, there was a separate email in the thread that talks about why having separate /usr is problematical in newer Linux distros so I may move off of having it separate.
Athena®, Created for the Cause™
Making a Difference in the Fight Against Breast Cancer
How and Why I Should Support Bottled Water!
Do not relinquish your right to choose bottled water as a healthy alternative to beverages that contain sugar, calories, etc. Your support of bottled water will make a difference! Your signatures count! Go to
http://www.bottledwatermatters.org/luv-bottledwater-iframe/dswaters and sign a petition to support your right to always choose bottled water. Help fight federal and state issues,
such as bottle deposits (or taxes) and organizations that want to ban the sale of bottled water. Support community curbside recycling programs. Support bottled water as a healthy way to maintain proper hydration. Our goal is 50,000 signatures. Share this petition
with your friends and family today!
Neal Rhodes
And the LVM GUI interface tells you it has to umount them, and when you ok that, of course it can't unmount them.
So, adding doesn't take a rescue boot. Only problem is I've already stuffed the chassis full, so I'm not likely to be adding drives. I guess I could replace the 1TB drive with a 2TB drive once they get to be $9 each next year.
Neal Rhodes
And had drive failures and it ran and booted fine on the single drive. Was able to replace and it regenerated.
Robert
>that they were in total smaller than the new 1TB disk I bought.
>
>And the LVM GUI interface tells you it has to umount them, and when you
>ok that, of course it can't unmount them.
>
>So, adding doesn't take a rescue boot. Only problem is I've already
>stuffed the chassis full, so I'm not likely to be adding drives. I
>guess I could replace the 1TB drive with a 2TB drive once they get to be
>$9 each next year.
Ah, yes, if you initially allocate all your space, add isn't that useful.
>From your initial post:
>So, I'm thinking of a layout like this:
>
> /dev/md0 on /boot type ext3 (rw) (whatever boot takes)
> /dev/md1 on / type ext3 (rw) (about 50GB)
> /dev/md2 on VolumeGroup00 (about 1TB)
> And logical volumes for /home and /u, which can be
> resized as needed between /home and /u
> /dev/sda? on /u2 (remaining 300GB,
> not Raid 1, just on the one bigger drive)
I'd start with
/dev/md0 on /boot type ext3 (rw) 500MB (red hat install default size)
/dev/md1 on VolumeGroup00 rest of disc
/dev/VolumeGroup00/h /home type ext3 10G (or whatever.. about 20% bigger
than current size)
/dev/VolumeGroup00/r / type ext3 10G (for gui system; 4G for headless)
/dev/VolumeGroup00/s swap 1 x ram size
/dev/VolumeGroup00/t /tmp type ext3 4G
/dev/VolumeGroup00/v /var type ext3 4G
I never try and allocate all the space.. I never know where I'll need it, and
having lots of available space lets me create new LVs or grow existing ones as
needed...
I almost always rename VolumeGroup00 to something else to make auto-complete
easier.. I often pick a single letter not starting any filename in /dev/, like
'e', 'g', etc..
James Sumners
> I almost always rename VolumeGroup00 to something else to make auto-complete
> easier.. I often pick a single letter not starting any filename in /dev/, like
> 'e', 'g', etc..
I like using /dev/lvm0/, /dev/lvm1/, etc.
--
James Sumners
http://james.roomfullofmirrors.com/
"All governments suffer a recurring problem: Power attracts
pathological personalities. It is not that power corrupts but that it
is magnetic to the corruptible. Such people have a tendency to become
drunk on violence, a condition to which they are quickly addicted."
Missionaria Protectiva, Text QIV (decto)
CH:D 59
Neal Rhodes
I screwed up my courage to the sticking point last night, exported the Virtualbox appliances, and reinstalled 6.2.
Of course, the first CD boot, to install, rattled for a while, then after a couple minutes it completely hung with the Centos logo on the screen. Great.
The 2nd time booting, using the text install, I didn't have the disk setup options. Super.
The third time I booted to the live CD, then picked "install to hard disk", and got the disk setup I've grown to know and.... be ok with.
I averaged out the responses, put /boot on /dev/md0, and the only PV on /dev/md1, and split out root, home and u under that. All appears to be well.
To answer your question, /u for me is typically my products, databases... all my company's stuff, which isn't mine personally. I attempt to segregate that from linux stuff in the standard directories. I attempt to avoid anything in /usr which isn't a part of Linux. Excepting /usr/local. So my priority in making backups is always /u first, then /home, then /usr/local, then / minus those three.
It's my hope that if I were to blow away the linux boot or / data, by having a separate filesystem I'm more likely to be able to pop the drive into a newly built replacement chassis and mount /home and /u. The junk in /usr/local is stuff I COULD reload if I had to. The stuff in /home and /u exists nowhere else on the planet but my backups.
Neal
LinuxGnome
> On Wed, Mar 21, 2012 at 13:54, Robert <r...@ale.spam.futz.org> wrote:
>> I almost always rename VolumeGroup00 to something else to make auto-complete
>> easier.. I often pick a single letter not starting any filename in /dev/, like
>> 'e', 'g', etc..
>
> I like using /dev/lvm0/, /dev/lvm1/, etc.
>
>
vg`hostname -s` and lvusr lvvar lvopt lvhome etc have served me well.
A good reason for renaming the system VG is having to do system recovery. If I pull my malfunctioning VG "system" from the box, and install it to another box with a VG "system", hilarity ensues!
mi...@trausch.us
> Maybe I'm doing it wrong, but I have found having /boot on a non-LVM
> partition and everything else managed by LVM to be quite useful. In
> fact, I had to resize a root partition on a server I was setting up just
> today (well, yesterday at this point) because I forgot to appease the
> great Oracle client with way too much swap space (I gave 512MB and it
> wanted twice RAM [60GB of swap?! Ridiculous]). It's also handy when you
> need to add a new drive into the mix for more free space.
Having a /boot on a "normal" partition, and using LVM/mdraid on the
others is just fine. It's a "legacy" setup in that it isn't strictly
speaking necessary these days (GRUB 2 can deal with the kernel and
initrd being on LVM and mdraid storage, for example, in most
situations). That said, it is always safe to have a separate /boot.
What I typically do is have a smaller drive be the boot drive, use
mdraid on whole disks in a single partition, and then use LVM on top of
the mdraid.
--- Mike
--
A man who reasons deliberately, manages it better after studying Logic
than he could before, if he is sincere about it and has common sense.
--- Carveth Read, “Logic”
mi...@trausch.us
> I've usually broken these out too.. And was surprised to find that Fedora 16
> spits out a warning if you have /usr on it's own partition. The seems to be
> something to do with the new systemd, which manages the init scripts.
This was just a huge set of threads on the gentoo-user mailing list.
The "right" solution is to have /usr separate from / and use / for the
host-local things. /usr can then be shared between many systems for
savings in administration. Then the only thing that you have to manage
per-each-host is the /etc tree and things like /var.
mi...@trausch.us
> Hmm. I can see having /var, /tmp, /usr/local, /opt, /home separate from
> /. I don't see a benefit splitting out /usr. I'm very curious as to the
> reasoning. In fact, I see a decent amount of reasoning to keep /sbin,
> /usr, /usr/sbin and /etc always on the same partition. Main one is that
> is what gets updated during a system upgrade.
The idea is that eventually, /bin and /sbin will be links to /usr/bin
and /usr/sbin, unifying the whole mess in /usr. This removes the
special casing for / that has to be done in the build systems of many
projects. (Not that this affects most system administrators; if they
build software on the system themselves, it usually goes to /usr/local
by default; but trying to manually graft things into /bin, /sbin, etc.
can be a pain.)
Now, you have a system where /etc and /boot are on the root filesystem,
and you have a shared /usr tree that many different systems can share.
You can have /var on a separate tree as well.
Note that these "modern" systems will use /run instead of /var/run, and
it expects that /run will be a tmpfs type thing.
I usually also have a filesystem for /srv, which is the filesystem that
I use to hold all data that services expose over a network. e.g., web
site roots, databases, and Samba filesystems are all in /srv on my systems.
mi...@trausch.us
> /usr because I’ve been doing it for years on other platforms as the UNIX
> System V way of doing things. I even create /opt although other than
> Dell OpenManage I’ve not found many Linux apps that use /opt.
Just proprietary (binary-only) stuff that you don't really mess with
should be in /opt. For example, that's where I install the Oracle JRE.
If I use /opt on a system, it is (usually) its own filesystem because
none of my boot process will ever depend on the things there.
> However, there was a separate email in the thread that talks about why
> having separate /usr is problematical in newer Linux distros so I may
> move off of having it separate.
You don't need to have /usr on / --- all that *has* to happen is that
/usr has to be mounted in the very early-boot process. You can
accomplish that pretty easily with an initrd that pre-mounts /usr before
starting the init daemon, regardless of what init daemon that is.
Ted W
Lightner, Jeff
Like everything else its opinion based. Mine is based on having seen many things fill up / when NOT in separate partitions and causing issues. By breaking things out I can easily monitor the various filesystems for when they are getting full and avoid the risk of corruption to key files that I’ve seen occur when / has gotten full.
You need to read the rest of the thread as I went into some detail as to why I split out certain filesystems and others posted about why /usr being separate going forward might be (or might not be) a bad idea.
From: ale-b...@ale.org [mailto:ale-b...@ale.org] On Behalf Of Ted W
Sent: Thursday, March 22, 2012 9:12 AM
To: Atlanta Linux Enthusiasts
Athena®, Created for the Cause™
Making a Difference in the Fight Against Breast Cancer
How and Why I Should Support Bottled Water!
Do not relinquish your right to choose bottled water as a healthy alternative to beverages that contain sugar, calories, etc. Your support of bottled water will make a difference! Your signatures count! Go to http://www.bottledwatermatters.org/luv-bottledwater-iframe/dswaters and sign a petition to support your right to always choose bottled water. Help fight federal and state issues, such as bottle deposits (or taxes) and organizations that want to ban the sale of bottled water. Support community curbside recycling programs. Support bottled water as a healthy way to maintain proper hydration. Our goal is 50,000 signatures. Share this petition with your friends and family today!
---------------------------------
Stephen R. Blevins
partitioning, but the author's take is that it ultimately depends on the
usage goals of the system in question. I know that's not much help, but
it is a place where the topic is discussed, however briefly.
Stephen R. Blevins
stephen....@gmail.com
> Ted W. < T...@Techmachine.net <mailto:T...@Techmachine.net> >
> Registered GNU/Linux user #413569
>
>
>
>
>
Lightner, Jeff
-----Original Message-----
From: ale-b...@ale.org [mailto:ale-b...@ale.org] On Behalf Of Stephen R. Blevins
Sent: Thursday, March 22, 2012 9:35 AM
To: Atlanta Linux Enthusiasts
Stephen R. Blevins
stephen....@gmail.com
Athena(r), Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer
---------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------
Leam Hall
environment has to meet compliance requirements and I believe that is
one of them. Don't ask me why, it just is.
For personal use /usr has always been the largest space hog so I can
move it to a larger partition easier if it is already separate. This is
mostly a habit as I started before LVM was around.
Leam