disksuite and veritas config
Wence Gutierrez
E450 with 6 disks across 2 controllers. A couple of sun techs suggest that
many admins are using DiskSuite to mirror the boot disk and Verities to
mirror data drives. The configuration we would like to use is to have the
OS on c0t0d0 mirrored to c1t0d0 and then use Verities to mirror c0t1d0-
c0t2d0 to c1t1d0-c1t2d0 on the second controller. I have mirrored the boot
disk without problem.
My question involve Verities. Verities wants to have at least one disk in
rootdg. Its my understanding that this can make recovery more difficult
because the disk are encapsulated. When talking to sun support, they
notified me that at least on disk must be in the rootdg group. Sun also
suggest that a small drive could be installed into the controller and
allowed to hold the place in rootdg.
Does anyone have any experience in using both DiskSuite and Verities for
mirroring. Is this preferable to using only Verities?
Any advice would be appreciated.
-Wence
Patrik Linder
Hello
There is no need to use disk suite, disk suite does not have any
advantages over Veritas.
If you encapsulate your rootdisk you can run vxunroot to get rid of
Veritas. I suggest that you put only the
two root disks in the rootdg an the rest in any other dg as you please.
This gives you the advantage to be
able to install Veritas on your system again if you wants to, or install
Veritas on any other machine.
When this is done you just have to attach the disks to the newly
installed system and Veritas finds an enables you to use the disks right
away
with out any changes to your configuration. the database localized on
every disk has information on what disk group it belonged to when last
used.
This is a very nice feature. If one disk in your system churches you can
fore example one 9GB disk you can have tree 3GB disks as "Spare" disks.
it will automatically move data from the faulted mirror to the hot
relocation disks, "Spare". It can do this because it does not use any
partitions on the disk, only logical subdisks.
Well I have to stop now, I hope you make the right choice an selects
only the Veritas product.
/
/Patrik Linder Dimension Sweden
Doug Hughes
> Looking for some advice about Verities and DiskSuite. I am configuring an
> E450 with 6 disks across 2 controllers. A couple of sun techs suggest that
> many admins are using DiskSuite to mirror the boot disk and Verities to
> mirror data drives. The configuration we would like to use is to have the
> OS on c0t0d0 mirrored to c1t0d0 and then use Verities to mirror c0t1d0-
> c0t2d0 to c1t1d0-c1t2d0 on the second controller. I have mirrored the boot
> disk without problem.
>
> My question involve Verities. Verities wants to have at least one disk in
> rootdg. Its my understanding that this can make recovery more difficult
> because the disk are encapsulated. When talking to sun support, they
> notified me that at least on disk must be in the rootdg group. Sun also
> suggest that a small drive could be installed into the controller and
> allowed to hold the place in rootdg.
>
We have 4 storage arrays, and I encapsulate the rootdisk into rootdg and
setup the other internal disk as a mirror. If the rootdg disk fails, the
machine will boot off the mirror (boot disk1) without so much as a fuss.
I have had this happen. You use vxdiskadm to replace the primary disk and
you're back to the races.
> Does anyone have any experience in using both DiskSuite and Verities for
> mirroring. Is this preferable to using only Verities?
>
It's largely religious. ;) People more familiar with disksuite opt to
do the internal mirroring with disksuite under the theory that it will
reduce complexity. If 'I' am going to install SEVM anyway, I mirror
the boot disk with it under the theory that I'm reducing complexity by
having only 1 volume management product on the system (and associated
kernel modules) vs 2.
> Any advice would be appreciated.
>
--
____________________________________________________________________________
Doug Hughes Engineering Network Services
System/Net Admin Auburn University
do...@eng.auburn.edu
FishinJT
>mirroring. Is this preferable to using only Verities?
>
We have a bunch of machines running the exact setup you describe. Couple of
items.
1) We mirror root with ODS... VM the rest...
2) Yes, you need at least one disk it Veritas's rootdg but.... why encasulate??
Do you have any data on the disks yet? Can you back them up if you do? I
would just initialize the drives to VM and put them in the rootdg. Create new
volumes and then restore the data if you had any.
3) Yes, VM can handle recovery of a failure ok but the procedure of upgrading
the OS and/or VM with a encapsulated root disk is way more complicated then if
you have root disks mirrored with ODS.
I love VM, can't image life without but when it comes to root disk I think ODS
is a better choice
HTH,
john
------------------------------
John Stachlewicz
Sys Admin
fish...@aol.com
------------------------------
Spike White
FishinJT wrote in message <19990415175706...@ng-fs1.aol.com>...
>3) Yes, VM can handle recovery of a failure ok but the procedure of
upgrading
>the OS and/or VM with a encapsulated root disk is way more complicated then
if
>you have root disks mirrored with ODS.
>
Sure it's more complicated, but it's not rocket science. And how often do
you upgrade
your OS anyway?
umount non-root fs's and comment them from /etc/vfstab
vxunroot
install_start
uninstall Veritas
upgrade Solaris
install Veritas
install_finish
Uncomment entries in /etc/vfstab
That's it. Not rocket science.
>I love VM, can't image life without but when it comes to root disk I think
ODS
>is a better choice
I thought so too, until Veritas 2.5/2.6. Now I appreciate having all my
disk
mgmt under one roof.
Spike
FishinJT
>you upgrade
>your OS anyway?
>
>umount non-root fs's and comment them from /etc/vfstab
>vxunroot
>install_start
>uninstall Veritas
>upgrade Solaris
>install Veritas
>install_finish
>Uncomment entries in /etc/vfstab
>
>That's it. Not rocket science.
I understand your point. It looks like a nice little check list but.... what
happens when things go wrong... Try and find anybody that can help you when
you have a problem with an encapsulated root disk when your upgrading your
global production box on a Sunday morning... Maybe somebody will call you
Monday afternoon... 8^)
Of course you need to also do this when you upgrade VM, which we do more often
then Solaris....
But then again... maybe time to revist VM for root disks....
Patrik Linder
>
> >Sure it's more complicated, but it's not rocket science. And how often do
> >you upgrade
> >your OS anyway?
> >
> >umount non-root fs's and comment them from /etc/vfstab
> >vxunroot
> >install_start
> >uninstall Veritas
> >upgrade Solaris
> >install Veritas
> >install_finish
> >Uncomment entries in /etc/vfstab
> >
> >That's it. Not rocket science.
>
> I understand your point. It looks like a nice little check list but.... what
> happens when things go wrong... Try and find anybody that can help you when
> you have a problem with an encapsulated root disk when your upgrading your
> global production box on a Sunday morning... Maybe somebody will call you
> Monday afternoon... 8^)
>
> Of course you need to also do this when you upgrade VM, which we do more often
> then Solaris....
>
> But then again... maybe time to revist VM for root disks....
>
> john
>
Veritas has 24h service....
you have to bee quite good to boot on one side of a miirored disk suite
root disk, since you lost 50% of your databases.
Then you might need some help...
:-D
/Patrik
Bruce Porter
>
> FishinJT wrote:
> > >Sure it's more complicated, but it's not rocket science. And how often do
> > >you upgrade
> > >your OS anyway?
> > >umount non-root fs's and comment them from /etc/vfstab
> > >vxunroot
> > >install_start
> > >uninstall Veritas
> > >upgrade Solaris
> > >install Veritas
> > >install_finish
> > >Uncomment entries in /etc/vfstab
> > >That's it. Not rocket science.
> > I understand your point. It looks like a nice little check list but.... what
> > happens when things go wrong... Try and find anybody that can help you when
> Then you might need some help...
> :-D
> /Patrik
Personally I reckon it is easier to upgrade the OS with Disksuite than
VM (and belive me I have done a few in the last couple of months) .
Given the choice I would use ODS on the root disk and VM for data
The upgrade steps for VM are not quite as staright forward as listed
above. There are 2 Sun docs that can be followed ( 14189 and 16708),
Mixed with some knowledge of VM command line usage it is fairly straight
forward , although time consuming.
--
Bruce
-----
Y2K SE
-----
martin...@par.univie.ac.at
>
> Personally I reckon it is easier to upgrade the OS with Disksuite than
> VM (and belive me I have done a few in the last couple of months) .
> Given the choice I would use ODS on the root disk and VM for data
as the ODS experts are listening, I'll add another question,
regarding upgrading Solaris on a mirrored root disk.
I usually prefer complete reinstalls to upgrades, and the ODS
docs only talk about upgrades. I assume the high level steps
involved in a OS reinstall (installing a newer OS release, of course)
would be:
- unmirror the root disk (that's explained in the docs)
- install new Solaris release
- install (and patch) ODS (the new release that comes with the new OS)
- mirror the root disk (as explained in the docs)
- If you had other metadevices before the reinstall, those
will be recognized by ODS again (as long as the only replicas
were not just on the root disk(s)).
this procedure looks much simpler that the upgrading described
in ODS docs. Is that how people actually do it ?
thanks in advance,
mp.
--
Martin Paul | Systems Administrator
Institute for Software Technology | mar...@par.univie.ac.at
and Parallel Systems | Tel: 01-3105608-84
Liechtensteinstrasse 22, A-1090 Wien | Fax: 01-3105608-88
Peter C. Tribble
martin...@par.univie.ac.at writes:
>
> as the ODS experts are listening, I'll add another question,
> regarding upgrading Solaris on a mirrored root disk.
>
> I usually prefer complete reinstalls to upgrades, and the ODS
> docs only talk about upgrades. I assume the high level steps
> involved in a OS reinstall (installing a newer OS release, of course)
> would be:
>
> - unmirror the root disk (that's explained in the docs)
> - install new Solaris release
> - install (and patch) ODS (the new release that comes with the new OS)
> - mirror the root disk (as explained in the docs)
> - If you had other metadevices before the reinstall, those
> will be recognized by ODS again (as long as the only replicas
> were not just on the root disk(s)).
>
> this procedure looks much simpler that the upgrading described
> in ODS docs. Is that how people actually do it ?
Doesn't work quite that way. What you need to do is read the disaster
recovery section of the DiskSuite documentation. Basically:
- reinstall Solaris, DiskSuite, etc
- create new metadevice databases, mirror root etc
- take a copy of the old /etc/opt/SUNWmd/md.cf (which should be an
accurate reflection of what the meta.db file would have looked like if
it had been fully maintained)
- use that to create the entries in meta.db for the devices you want
to recover, converting all the mirrors to one-way. Probably just append
the saved md.cf to meta.db and edit.
- re-run metainit to recreate all the metadevices (With a raid-5, make sure
you've got the -r flag!)
- resync all the mirrors
All very quick and painless. The only trick with a reinstall is to make
sure that the device mapping doesn't change, so that the disk names
haven't changed. I presume you could just change the names to point to
the correct devices before running metainit. There's probably a
shortcut, but the disaster recovery method is easy enough.
--
-Peter Tribble
HGMP Computing Services
http://www.hgmp.mrc.ac.uk/~ptribble/
Doug Hughes
> >Sure it's more complicated, but it's not rocket science. And how often do
> >you upgrade
> >your OS anyway?
> >
> >umount non-root fs's and comment them from /etc/vfstab
> >vxunroot
> >install_start
> >uninstall Veritas
> >upgrade Solaris
> >install Veritas
> >install_finish
> >Uncomment entries in /etc/vfstab
> >
> >That's it. Not rocket science.
>
> I understand your point. It looks like a nice little check list but.... what
> happens when things go wrong... Try and find anybody that can help you when
> global production box on a Sunday morning... Maybe somebody will call you
> Monday afternoon... 8^)
>
Well, there's always ssa-managers and veritas-users mail lists.
Besides, there's even simpler ways to do it (from a seeing what's
going on point of view - install_start and install_finish hide a lot
of details)
> Of course you need to also do this when you upgrade VM, which we do more often
> then Solaris....
>
> But then again... maybe time to revist VM for root disks....
>
--
Kelly Fergason
I think you are making it more complicated than it really is.
First, save the md.tab file.
For a new install, install the OS, install ODS, using the md.tab
file you saved. (assuming devices haven't changed)
You don't have to worry about unmirroring root, as you are
doing a re-install.
For an upgrade, uninstall ODS, upgrade the OS, install ODS. Same
comment about the md.tab.
It really is that simple.
Kelly
On 19 Apr 1999 14:27:23 GMT, martin...@par.univie.ac.at wrote:
>Bruce Porter <bruce....@bt.smeg.com> wrote:
>>
>> Personally I reckon it is easier to upgrade the OS with Disksuite than
>> VM (and belive me I have done a few in the last couple of months) .
>> Given the choice I would use ODS on the root disk and VM for data
>
>as the ODS experts are listening, I'll add another question,
>regarding upgrading Solaris on a mirrored root disk.
>
>I usually prefer complete reinstalls to upgrades, and the ODS
>docs only talk about upgrades. I assume the high level steps
>involved in a OS reinstall (installing a newer OS release, of course)
>would be:
>
>- unmirror the root disk (that's explained in the docs)
>- install new Solaris release
>- install (and patch) ODS (the new release that comes with the new OS)
>- mirror the root disk (as explained in the docs)
>- If you had other metadevices before the reinstall, those
> will be recognized by ODS again (as long as the only replicas
> were not just on the root disk(s)).
>
>this procedure looks much simpler that the upgrading described
>in ODS docs. Is that how people actually do it ?
>
>thanks in advance,
>
>mp.
FishinJT
Yes they do and so does Sun... but... mention "encapsulated root disk" and
suddenly the phone will go silent for a while, then you will get a depressed
sounding "Oh... from the other end. Any time I've had to call for support to
Veritas on a Sunday morning (7 a.m. EST) I've had to wait for them to track
down the on-call engineer and then wait a couple of hours for him to call
back... Hmm... can't imagine what kind of response I'ld get if I did have an
encapsulated root disk....
martin...@par.univie.ac.at
>
> - reinstall Solaris, DiskSuite, etc
> - create new metadevice databases, mirror root etc
> - take a copy of the old /etc/opt/SUNWmd/md.cf (which should be an
> accurate reflection of what the meta.db file would have looked like if
> it had been fully maintained)
> - use that to create the entries in meta.db for the devices you want
> to recover, converting all the mirrors to one-way. Probably just append
> the saved md.cf to meta.db and edit.
> - re-run metainit to recreate all the metadevices (With a raid-5, make sure
> you've got the -r flag!)
> - resync all the mirrors
Hm I still don't get it. Maybe I'm thinking too complicated ..
let's assume a simple setup, 4 harddisks,
c0t0: boot disk, metadb on seperate slice
c0t1: mirror of boot disk, metadb on seperate slice
c0t2: one slice, data, metadb on the same slice as data
c0t3: mirror of c0t2, metadb on the same slice as data
Now if I reinstall the OS and disksuite I lose the information
about where the metadb's are (on c0t[23]).
I thought the replicas on the data disks which are not touched
during reinstall would save the metadevice information, so no
need to recreate them. obviously that's not true. That's because
the entries in /etc/system and mddb.cf get lost during the
reinstall.
how am I to tell ODS that there are replicas on c0t[23] after
the reinstall ? I can't create them from scratch without losing
the data as far as I understand it. do I have to add them to
the md.tab before running metainit -a, too ?
so my problem is that I think that I have not lost the whole
ODS configuration (the replicas on the data disks are still there),
but that's what the disaster recovery talks about. For me the
problem sounds like "I've lost the information where the replicas
are, but there still are some" -- and this question is not answered
in the docs.
Kelly Fergason
The manual does mention saving the mddb entry in the md.tab.
>
>how am I to tell ODS that there are replicas on c0t[23] after
>the reinstall ? I can't create them from scratch without losing
>the data as far as I understand it. do I have to add them to
>the md.tab before running metainit -a, too ?
>
>so my problem is that I think that I have not lost the whole
>ODS configuration (the replicas on the data disks are still there),
>but that's what the disaster recovery talks about. For me the
>problem sounds like "I've lost the information where the replicas
>are, but there still are some" -- and this question is not answered
>in the docs.
>
>mp.
Ok, lets back up a step. Why are you trying to save the info from the
previous install? You are doing a brand new install. Yes, the info
on the replicas would be the same, but who cares? re-make the
replica. Its part of the ODS install. Its one command. You did
save the info before started the re-install?
Kelly
Peter C. Tribble
martin...@par.univie.ac.at writes:
> Peter C. Tribble <ptri...@hgmp.mrc.ac.uk> wrote:
>>
>> - reinstall Solaris, DiskSuite, etc
>> - create new metadevice databases, mirror root etc
>> - take a copy of the old /etc/opt/SUNWmd/md.cf (which should be an
>> accurate reflection of what the meta.db file would have looked like if
>> it had been fully maintained)
>> - use that to create the entries in meta.db for the devices you want
>> to recover, converting all the mirrors to one-way. Probably just append
>> the saved md.cf to meta.db and edit.
>> - re-run metainit to recreate all the metadevices (With a raid-5, make sure
>> you've got the -r flag!)
>> - resync all the mirrors
>
> Hm I still don't get it. Maybe I'm thinking too complicated ..
>
> let's assume a simple setup, 4 harddisks,
>
> c0t0: boot disk, metadb on seperate slice
> c0t1: mirror of boot disk, metadb on seperate slice
> c0t2: one slice, data, metadb on the same slice as data
> c0t3: mirror of c0t2, metadb on the same slice as data
OK, so to recover you need to run metadb to recreate the metadbs, and
then metainit to recreate the metadevices. Just create everything the
same way you did before, and all the devices will come back. (Don't
forget the -r on raid-5 devices.)
> Now if I reinstall the OS and disksuite I lose the information
> about where the metadb's are (on c0t[23]).
And other information as well. I presume it's possible to restore all
the information required back into the right places, but starting over
isn't so bad.
> I thought the replicas on the data disks which are not touched
> during reinstall would save the metadevice information, so no
> need to recreate them. obviously that's not true. That's because
> the entries in /etc/system and mddb.cf get lost during the
> reinstall.
Even putting the /etc/system entries back doesn't work. As I said,
presumably if you put everything back in place correctly it would all
work. (Is md.cf enough, I wonder?)
> how am I to tell ODS that there are replicas on c0t[23] after
> the reinstall ? I can't create them from scratch without losing
> the data as far as I understand it. do I have to add them to
> the md.tab before running metainit -a, too ?
If you're going to recreate the configuration, you don't need to worry
about the contents of the old metadbs. Just create new ones in the same
places.
> so my problem is that I think that I have not lost the whole
> ODS configuration (the replicas on the data disks are still there),
> but that's what the disaster recovery talks about. For me the
> problem sounds like "I've lost the information where the replicas
> are, but there still are some" -- and this question is not answered
> in the docs.
The disaster recovery is basically "what do I do if the system disk gets
wiped out" which is a reasonable approximation of what a reinstall
does.
I've been through this several times recently, reinstalling servers
with Solaris 7 (sometimes onto new disks, as it requires rather more space,
especially in 64-bit mode, that 2.5.1 did). It took a little
head-scratching at first to realise that it was the disaster recovery
procedure that I needed!
martin...@par.univie.ac.at
> Ok, lets back up a step. Why are you trying to save the info from the
> previous install? You are doing a brand new install. Yes, the info
> on the replicas would be the same, but who cares? re-make the
> replica. Its part of the ODS install. Its one command. You did
> save the info before started the re-install?
yes, it'd be a brand new OS install, but on extra disks I have
existing metadevices, and existing replicas. so the question was
how to create replicas on partitions which already contain data
(a setup where the replicas are NOT in seperate partitions). The
answerbook made me believe that you can create replicas only
on empty partitions. so obviously that's not true. Here' what I
tried now (c0t0 is irrelevant for this example):
# metadb
flags first blk block count
a m p luo 16 1034 /dev/dsk/c0t0d0s7
a p luo 1050 1034 /dev/dsk/c0t0d0s7
a u 16 1034 /dev/dsk/c0t1d0s0
a u 1050 1034 /dev/dsk/c0t1d0s0
a u 16 1034 /dev/dsk/c0t1d0s1
a u 1050 1034 /dev/dsk/c0t1d0s1
# metastat -p
...
d100 1 2 c0t1d0s0 c0t1d0s1 -i 32b
# mount /dev/md/dsk/d100 /mnt
# df -k /mnt
Filesystem kbytes used avail capacity Mounted on
/dev/md/dsk/d100 4355642 52145 4259941 2% /mnt
# umount /mnt
# metadb -d c0t1d0s0 c0t1d0s1
# metadb
flags first blk block count
a m p luo 16 1034 /dev/dsk/c0t0d0s7
a p luo 1050 1034 /dev/dsk/c0t0d0s7
# metadb -a -c 2 c0t1d0s0 c0t1d0s1
(!! This is what I thought wouldn't work !!)
# metadb
flags first blk block count
a m p luo 16 1034 /dev/dsk/c0t0d0s7
a p luo 1050 1034 /dev/dsk/c0t0d0s7
a u 16 1034 /dev/dsk/c0t1d0s0
a u 1050 1034 /dev/dsk/c0t1d0s0
a u 16 1034 /dev/dsk/c0t1d0s1
a u 1050 1034 /dev/dsk/c0t1d0s1
# mount /dev/md/dsk/d100 /mnt
# df -k /mnt
Filesystem kbytes used avail capacity Mounted on
/dev/md/dsk/d100 4355642 52145 4259941 2% /mnt
This is what the answerbook says about creating replicas:
You can create state database replicas on either a dedicated slice,
or on a slice that will be used as part of a simple metadevice,
RAID5 metadevice, or trans metadevice.
A state database replica cannot be part of root (/), swap, /usr, an
existing file system, or a slice containing data.
obviously it works anyway as the space for the replica is already
reserved on disk.
hmmm, this opens another question -- what if in the next release of
ODS the size of the replica grows (not 1034 sectors anymore) ??
Maybe the safest way is to always put replicas on separate partitions,
sized big enough for an eventual growth.