ZFS on top VxVM
Wolfgang
think this is optimal;-). But is there a way to enlarge the zfs, when
the volume grows?
Thanks
Wolfgang
Ian Collins
> ZFS again: i have build a Veritas Volume and set a zfs on top (i dont
> think this is optimal;-).
>
> But is there a way to enlarge the zfs, when the volume grows?
If ZFS uses the raw devices, you can add new devices to the pool.
--
Ian Collins.
Darren Dunham
You'll probably need to export/import the pool before it will recognize
the change in size.
--
Darren Dunham ddu...@taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
Wolfgang
I just tried it, because we use veritas VM and Cluster and i dont want
mix different volume managers, but I want use zfs. But the question is
somehow generally, e.g. use a enlarged disk on a HW Raid.
So adding devices also not very elegant.
W.
Ian Collins
adding more components. If you pool is a single volume, there's not a
lot you can do.
--
Ian Collins.
lahuman9
ZFS is a volume manager too... Is this a supported configuration with
ZFS
on top of VM, and then with VCS?
I would not want to be the one calling up Symantec or Sun when
something
is down at 3am...
Darren Dunham
> You can enlarge a zpool by replacing a component with a larger one or by
> adding more components. If you pool is a single volume, there's not a
> lot you can do.
Enlarging the single component should work as well:
I created a 0.2G VMware disk:
# zpool create tank /dev/dsk/c0d1
# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
tank 192M 89K 192M 0% ONLINE -
# zpool export tank
Enlarge the disk with vdiskmanager to 0.5G... boot...
In Solaris use fdisk to delete the EFI partition, then recreate at 100%
of disk.
In 'format', nuke any existing slices from 0-6 and create a new one on
slice 0 taking up the whole disk.
# zpool import tank
# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
tank 499M 92K 499M 0% ONLINE -
So the Solaris steps to use the enlarged space aren't the simplest, but
once complete, ZFS will recognize the space and use it on next import.
usenetper...@gmail.com
> Ian Collins <ian-n...@hotmail.com> wrote:
> > You can enlarge a zpool by replacing a component with a larger one or by
> > adding more components. If you pool is a single volume, there's not a
> > lot you can do.
So it seems.
> Enlarging the single component should work as well:
> I created a 0.2G VMware disk:
> # zpool create tank /dev/dsk/c0d1
> # zpool list
> NAME SIZE USED AVAIL CAP HEALTH ALTROOT
> tank 192M 89K 192M 0% ONLINE -
> # zpool export tank
> Enlarge the disk with vdiskmanager to 0.5G... boot...
> In Solaris use fdisk to delete the EFI partition, then recreate at 100%
> of disk.
> In 'format', nuke any existing slices from 0-6 and create a new one on
> slice 0 taking up the whole disk.
> # zpool import tank
> # zpool list
> NAME SIZE USED AVAIL CAP HEALTH ALTROOT
> tank 499M 92K 499M 0% ONLINE -
> So the Solaris steps to use the enlarged space aren't the simplest, but
> once complete, ZFS will recognize the space and use it on next import.
Hmm Ill look into export/import but Im wondering a bit what happens
to the data in all this...
So how about working with a SAN then... From the SAN side you need to
grow a LUN
because you are running out of space for Oracle say. Easy as cake.
Things churn away
and you have a larger LUN. Trouble is the Sun label still thinks you
have X GB instead of X+Y.
If you do run a format the ascii description of total space available
has been updated but prtvtoc etc still see the label unchanged.
Because it IS unchanged. Urk!
I cannot see how to tell the OS side: things have grown (or where
if there are multiple partitions on the LUN) without backing the lot
up and nuking the ZFS or
UFS label and reformatting and possibly re-partitioning in the case of
UFS or setting
quotas in the case of ZFS.
Sure you can catonate a new LUN onto the old one to make it bigger,
but from the SAN point of view that's a kludge isnt it?
Guess there's no such animal as a "dynamic label" : >
Darren Dunham
> Hmm Ill look into export/import but Im wondering a bit what happens
> to the data in all this...
It's all there. The import would fail otherwise.
> So how about working with a SAN then... From the SAN side you need to
> grow a LUN
> because you are running out of space for Oracle say. Easy as cake.
> Things churn away
> and you have a larger LUN. Trouble is the Sun label still thinks you
> have X GB instead of X+Y.
Right. That's why I had to rewrite the label. But since none of the
data is modified by that action, it was successful.
> If you do run a format the ascii description of total space available
> has been updated but prtvtoc etc still see the label unchanged.
> Because it IS unchanged. Urk!
> I cannot see how to tell the OS side: things have grown (or where
> if there are multiple partitions on the LUN) without backing the lot
> up and nuking the ZFS or
> UFS label and reformatting and possibly re-partitioning in the case of
> UFS or setting
> quotas in the case of ZFS.
You have to write a new label. You don't have to back anything up (at
least not if you're using EFI labels)
On x86 only:
fdisk /dev/rdsk/<whatever>
delete partition (should only be one)
create partition (type EFI)
should default to 100% of (now larger) disk
On all systems:
format -e
select correct disk
label
type EFI
partition
print (verify disk is correct size)
remove any existing slices in the 1-7 range
add a slice 0. Start cyl 34, maximum size
label
type EFI
> Guess there's no such animal as a "dynamic label" : >
I believe they're working on it, because that's really the only thing
that makes this process difficult.
usenetper...@gmail.com
> usenetpersonger...@gmail.com wrote:
> > Hmm Ill look into export/import but Im wondering a bit what happens
> > to the data in all this...
> It's all there. The import would fail otherwise.
>
> > So how about working with a SAN then... From the SAN side you need to
> > grow a LUN
> > because you are running out of space for Oracle say. Easy as cake.
> > Things churn away
> > and you have a larger LUN. Trouble is the Sun label still thinks you
> > have X GB instead of X+Y.
> Right. That's why I had to rewrite the label. But since none of the
> data is modified by that action, it was successful.
I have done this before and yes it has worked. But Im thinking about
an Oracle production environment here. Downtime is nearly impossible
to get..
Unless you are a dba of course. But systems people arent allowed to do
ANYTHING : >
Patches?? fergetabout it
Point is the SAN can expand and not affect a running system.
But the label is unchanged. Seems like everything comes to a
screeching halt
due to that fact. In the case of EFI where you have one partition and
then the
reserved area it doesnt seem all that hard to implement really.
But to be safe for now it seems that you need to shut everything down
after an RMAN backup, newfs
or zpool destroy/create and then restore. Pretty ugly!! And a hard
sell to the
production types.
> > If you do run a format the ascii description of total space available
> > has been updated but prtvtoc etc still see the label unchanged.
> > Because it IS unchanged. Urk!
> > I cannot see how to tell the OS side: things have grown (or where
> > if there are multiple partitions on the LUN) without backing the lot
> > up and nuking the ZFS or
> > UFS label and reformatting and possibly re-partitioning in the case of
> > UFS or setting
> > quotas in the case of ZFS.
>
> You have to write a new label. You don't have to back anything up (at
> least not if you're using EFI labels)
Yes. attempts to do so (a new label) on a ZFS'd SAN made LUN failed
today. Its not
a mirror, its ZFS on top of a RAID5 LUN actually in this case. Why?
Thats another story.
But ZFS or UFS its still the same problem.
> On x86 only:
> fdisk /dev/rdsk/<whatever>
> delete partition (should only be one)
> create partition (type EFI)
> should default to 100% of (now larger) disk
> On all systems:
> format -e
> select correct disk
> label
> type EFI
> partition
> print (verify disk is correct size)
> remove any existing slices in the 1-7 range
> add a slice 0. Start cyl 34, maximum size
> label
> type EFI
I guess you have tried this on a live system? I have no SAN in my
little lab
to test this out unfortunately. Wont be back on site till next week.
> > Guess there's no such animal as a "dynamic label" : >
> I believe they're working on it, because that's really the only thing
> that makes this process difficult.
Particularly if say there is several LUNS and partitions in some LUNS
(which could happen) Which one(s) grows()??
Yup. Sounds like "project" alright...
In a way you'd think the SAN vendor would support that already.
Maybe its harder to do than I think : >
Wolfgang
you are completely right, but the support problem at 3 a.m is still
there if i have a "supportet config" ;-(, as every time you have a
problem with interacting parts from different manufacturers.
we have build our own veritas resource-type ZFSvol and Zone, to use
Zones and ZFS in Cluster, because we can not use (by this
point)SunCluster, which btw. unfortunally do not support zfs on shared
Cluster-Devices.
So at the moment I testing, if i can use zfs as replacement for
vxvm/vxfs, but it still lacks some features of vx. but the main problem
is, that our backup-software dont support zfs in the used version (only
the newst version support zfs), so in respect to support it becomes
possible just in the last weeks.
This bring up some more and old question:
will be ZFS a Cluster-FS in future and that brings up the question what
happens to ZFS in respect to "Lustre".
another question is: in our tests we corrupted zfs by forced import.
Under ufs i have the possibility to make a fscheck and recover some of
the data, but zfs seems to dont give me a chance and ALL data are lost,
or is there a chance to have only some data corruptet, like the
nocheck-option on import.
thanks
WOlfgang
Darren Dunham
> I guess you have tried this on a live system? I have no SAN in my
> little lab
> to test this out unfortunately. Wont be back on site till next week.
I don't have any "play with" SAN available to me either. I mentioned
that this was all done on a VMware system.
>> > Guess there's no such animal as a "dynamic label" : >
>> I believe they're working on it, because that's really the only thing
>> that makes this process difficult.
>
> Particularly if say there is several LUNS and partitions in some LUNS
> (which could happen) Which one(s) grows()??
None of the partitions should grow. Any existing partition must remain
the same before and after the operation. You can enlarge any VTOC or
EFI slice on the fly afterward. That's not a problem.
I'm assuming the same can be done with MBR partitions as well, but I'm
less certain on that one.
> Yup. Sounds like "project" alright...
> In a way you'd think the SAN vendor would support that already.
The SAN vendor would need to understand all vendor disk labels.
Honestly if I was such a vendor, I'd probably ignore everything other
than Windows.
> Maybe its harder to do than I think : >
From the SAN/DISK side, I'd think it impossible due to the variety of
formats and host interactions. From the host itself (Solaris), I don't
think so. Really all the pieces seem to be there, it just needs to be
tied together into something easier for mortals to use and instrument.
The only piece that seems to require downtime is the fact that ZFS won't
reexamine the "disk" it's using until import. So the pool has to be
given an export/import to see the new size. Everything else should be
doable online. If you were mirroring, then even that wouldn't require
downtime since you could drop and add each side, one at a time.
usenetper...@gmail.com
> usenetpersonger...@gmail.com wrote:
> > I guess you have tried this on a live system? I have no SAN in my
> > little lab
> > to test this out unfortunately. Wont be back on site till next week.
> I don't have any "play with" SAN available to me either. I mentioned
> that this was all done on a VMware system.
I guess we'd both like to have a mini-SAN. Unfortunately despite
numerous
eBay probes nothing 'affordable' has ever come up when Ive looked.
> >> > Guess there's no such animal as a "dynamic label" : >
> >> I believe they're working on it, because that's really the only thing
> >> that makes this process difficult.
According to the guy I report to at one site some OS's he works with
can do this. He was surprised Solaris couldnt. I was sceptical given
Santricity simply presents you with an unlabeled LUN...
Which means format has to do the labelling and once that happens you
cant expand it
dynamically unless its a metadevice.
Can't use ZFS even though simple benchmarks show
its 2-4 times faster with writes over UFS. Too bad.
......
> > Yup. Sounds like "project" alright...
> > In a way you'd think the SAN vendor would support that already.
> The SAN vendor would need to understand all vendor disk labels.
> Honestly if I was such a vendor, I'd probably ignore everything other
> than Windows.
Well that's IT today yes?
Actually for quite some time.
> The only piece that seems to require downtime is the fact that ZFS won't
> reexamine the "disk" it's using until import. So the pool has to be
> given an export/import to see the new size. Everything else should be
> doable online. If you were mirroring, then even that wouldn't require
> downtime since you could drop and add each side, one at a time.
Same with UFS. So it seems Im stuck with metadevices and growfs -
which means I have to re-slice
the OS disks : > Argh.
ITguy
> Zones and ZFS in Cluster, because we can not use (by this
> point)SunCluster, which btw. unfortunally do not support zfs on shared
> Cluster-Devices.
ZFS is supported on as a fail-over file system in Sun Cluster 3.2
using the provided HAStorage agent. Are you looking for multiple
hosts to have concurrent read/write access to the same file system?
If so, ZFS does not meet your requirement. What exactly are you
trying to achieve?
Darren Dunham
> Which means format has to do the labelling and once that happens you
> cant expand it
> dynamically unless its a metadevice.
I don't see what metadevice has to do with it. I can usually expand the
last (or only) partition on the device. SVM just allows you to create a
new partition and attach it to existing storage.
> Can't use ZFS even though simple benchmarks show
> its 2-4 times faster with writes over UFS. Too bad.
Is this related to the expansion issue?
>> The only piece that seems to require downtime is the fact that ZFS won't
>> reexamine the "disk" it's using until import. So the pool has to be
>> given an export/import to see the new size. Everything else should be
>> doable online. If you were mirroring, then even that wouldn't require
>> downtime since you could drop and add each side, one at a time.
>
> Same with UFS. So it seems Im stuck with metadevices and growfs -
> which means I have to re-slice
> the OS disks : > Argh.
How does SVM get you any functionality increase over straight slices or
ZFS? You still have to bump the label, right?