Question: does that imply that you'll have more zfs filesystems
than you did ufs filesystems?
If so, WHY would you have lots of separate filesystems?
Heck, maybe a tree of them, I suppose.
----
Like they do for eg WD-40, have lists of the thousands of useful
things you can do with the product, just what benefit from having
lots of zfs's -- benefits (with much less pain, I guess) than
doing so with ufs filesystems.
Thanks!
David
Because you can.
Because they are easy to manage, no more buggering about with slices.
I use filesystems with different properties for different tasks. for
example Compressed for general data, uncompressed for already compressed
data. I also have different snapshot and replication routines based on
the nature of the data in a filesystem.
--
Ian Collins
The other major user of filesystems in OS is boot environments -
upgrading using "image-update" creates new ZFS filesystems. No
buggering about (TM) with slices and all that stuff, and now I can keep
6 bootable old versions of the OS if I want.
--
Chris
You need to think about zfs file systems a little differently than a
traditional ufs file system.
I describe zfs file systems as a single point of administration.
The administration being the types of things Ian described.
> In one of the either sun or commercial books on solaris 10 or opensolaris,
> it says how great it is that making a zfs a lightweight task (easy),
> because you'll likely have lots of them.
If it said that, it had things backwards. *Because* filesystems in ZFS
are so lightweight (and because they are not tied to underlying volumes
in the way (most? all?) other filesystems are) you can have lots of
them.
> If so, WHY would you have lots of separate filesystems?
A ZFS filesystem is not at all like a UFS filesystem, it's really just
a special directory to which you can attach administrative properties,
and which you can snapshot / clone easily (this is not quite true, hard
links can't cross these things, so they are more than special
directories).
Once you get the hang of ZFS filesystems being these lightweight
things, it completely changes the way you think about managing things.
Sun (surprisingly!) have understood this: look at the way Live Upgrade
works with ZFS roots.
some blueprints or recipies suggest this, even doing stuff like having a
ZFS filesystem for each user a system.
> If so, WHY would you have lots of separate filesystems?
it sounds retarded to me too. The less filesystems you have to deal with,
the better.
Anytime you see a df that needs |more you're dealing with some
trainwreck or a 1990s server at a university.
sun even warns that having thousands of ZFS things mounted at once is
slow. They never truly give a good reason why you'd want to do that in the
first place though.
It's not like all users are going to get different compression settings or
something like that.
But hey "the network is the computer" or something like that.
> it sounds retarded to me too. The less filesystems you have to deal with,
> the better.
Sure, but there is delegation, so you don't have to deal with anything,
aside from creating the file system.
> It's not like all users are going to get different compression settings or
> something like that.
No, but they might want snapshots, for example. So you create a file
system, give the appropriate permission to the user, possibly set a quota
and leave him be.
--
.-. .-. Yes, I am an agent of Satan, but my duties are largely
(_ \ / _) ceremonial.
|
| da...@fly.srk.fer.hr
Make that "used to work", it's FUBAR in update 8...
--
Ian Collins
You guys know SO MUCH about zfs, etc -- how did you
learn it all?>
Go to (expensive) sun class on it?
Go to user-group meetings?
Learn it all on your own, by reading man pages and other sun doc?
Any suggestions on what else to read?
And, were YOU writing a book or set-of-hints or whatever on
"how to use zfs", what kind of things would you say differently
from how eg sun said it?
What kinds of things would you say that they didn't cover at
all (or maybe covered, but put in some location you'd never
likely look in)?
What's really needed, at least for someone like me who works
totally on his own (am the only person I know who even uses Sun
or Solaris), is a lot of hand-holding kind of doc.
Maybe like an O'Reilly "Cookbook" -- "ZFS COOKBOOK".
Anything you guys (and others you could suggest it to) could
write-up here, that could give people like me (SCARED people
like me) (a) more confidence in knowing what they're doing,
(b) haveing some "cookbook"-like recipes with answers to common
(not problems, but) situations.
Seems to me that this zfs is the biggest change Sun has EVER
made to their systems, bigger even than going from sunos to sys-V/Solaris.
Like, if these zfs-things are like directories, than you can have
one inside another? Or not?
And if you do, how does that impact making backups, eg on tape.
Would there be boundaries within that tree, that would
limit/or-add to how snapshots worked?
-----
What would be nice is a BUNCH of individual "Here's how *I* did it, switching
from ufs to zfs, switching my backup procedures -- and if I had to do
it again, I'd do it this other way, and why.:
Whatever you'd write would likely end up in a zfs-tutorial faq, "owned"
in some sense by comp.unix.solaris.
Thanks so much for your replies thus far, in these many (repetitive)
HELP! posts I've made so far (and for those I've yet to make!).
-----
I'll say this so far: this zfs does seem pretty cool, especially
the zpools, letting zfs-internals take care of all that
disk stuff automatically.
(Again -- it's generally recommended to have no multiple-partition disks,
am I understanding that correctly? Each added disk adds its many gigabytes
all together as one bunch, as part of one zpool -- that's the recommended
usage?)
Thanks,
David
By using it (and reading some of the code) from day one!
> Any suggestions on what else to read?
The best place to start is:
http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
Then look at the archives of the ZFS discuss list on opensolaris.org.
The Read the official docs at docs.sun.com, they are very good.
> Like, if these zfs-things are like directories, than you can have
> one inside another? Or not?
Yes, filesystems can be nested.
> And if you do, how does that impact making backups, eg on tape.
That depends how you backup.
> (Again -- it's generally recommended to have no multiple-partition disks,
> am I understanding that correctly? Each added disk adds its many gigabytes
> all together as one bunch, as part of one zpool -- that's the recommended
> usage?)
Basically, yes. Give ZFS whole drives if you can.
--
Ian Collins
> Make that "used to work", it's FUBAR in update 8...
I've used LU on a U8 system in the last few days - what's broken as it
worked for me.
> some blueprints or recipies suggest this, even doing stuff like having a
> ZFS filesystem for each user a system.
Until U8 that was the only way to enforce quotas. However it's still a
good idea, I think (bear in mind that most systems have a very small
number of users, often with names like "oracle").
> Learn it all on your own, by reading man pages and other sun doc?
Pretty much that.
> And, were YOU writing a book or set-of-hints or whatever on
> "how to use zfs", what kind of things would you say differently
> from how eg sun said it?
I don't know how Sun present it really (which is mad since I work for
them at present), but I'd do two things:
* ZFS root is the key thing: ZFS is a much more compelling story for
system disks than it is for data.
* Snapshots and cloning change the world completely: if you want a
backout for a change, just create a snapshot, it's quick and cheap.
Suddenly instead of needing a spare pair of system disks and an hour or
so, you can make a bootable snapshot of a system in 10 seconds, most of
which is taken up by typing.
--tim
> I've used LU on a U8 system in the last few days - what's broken as it
> worked for me.
In fact, while writing the previous message, I patched a (virtual) U8
machine with LU.
The new userquota facility in U8 is going to be a real winner. It
removes the need for having tons of zfs filesystems, one per user as
someone else stated.
Now we can create one filesystem mypool/users
Set an overall quota on the filesystem
zfs set quota=100g mypool/users
and then set quotas per user in the filesystem. It is a great new
addition to zfs.
> The new userquota facility in U8 is going to be a real winner. It
> removes the need for having tons of zfs filesystems, one per user as
> someone else stated.
And for larger outfits, like the one I work for (web hosting), it
has not been an option at all. We would have needed filesystems in
the tens of thousands at least.
--
Usenet is not a right. It is a right, a left, a jab, and a sharp
uppercut to the jaw. The postman hits! You have new mail.
--
Ian Collins
> Do you have ZFS root with zones?
Yes. My "production" system has ZFS root but no zones. I've used my
scratch system (which has zfs root) to test zone patching as well.
This is all for patching rather than wholesale upgrade though, and
obviously my experience with U8 is pretty limited so far. As best I
can tell it all seems the same as U7 though.
--tim
I'd be interested to see if you can live upgrade from update 7 to update
8 on the system with ZFS root and zones.
--
Ian Collins
I'm sure he hasn't. I can't imagine LU to work with zfs root and
zones. There are too many errors generated. I'm happy to run on s10u8
now, but you know how much effort it has costed me. I still am not able
to patch safely because from within update 8 I can not create a good new
ABE to patch. Still waiting for a Live Upgrade patch that does work.
--
Dick Hoogendijk -- PGP/GnuPG key: 01D2433D
+ http://nagual.nl/ | SunOS 10u8 10/09 | OpenSolaris 2010.02 b123
+ All that's really worth doing is what we do for others (Lewis Carrol)
I disagree here. ZFS filesystems cost kernel memory plus the fact the
system boots up more slowly, the more zfs filesystems are present. They
are working on it, I heard though.
Yes -> no zones (?)
So do you or do you not have a zfs root system with zones installed and
did you try to Live Upgrade a ZFS root with zones system yet?
--
Dick Hoogendijk -- PGP/GnuPG key: 01D2433D
+ http://nagual.nl/ | SunOS 10u8 10/09 | OpenSolaris 2010.02 b125
> Yes -> no zones (?)
The "production" instance has no zones, the test instance does. (I put
"production" in quotes as its only production for me - runs source
control system &c, while the test instance is the one I use for
testing, which this week was (coincidentally) testing zone patching.
Can someone describe the errors they are seeing? How do you have zones
configured?
> I'd be interested to see if you can live upgrade from update 7 to
> update 8 on the system with ZFS root and zones.
I'd not tried this, but I have now. Basic process (this is on an x86 system):
* 10u7 install from media, ZFS root.
* Make a zone (ZFS root fs for zone)
* Remove 10u7 LU packages (SUNWlucfg, SUNWlur, SUNWluu, SUNWluzone),
add 10u8 packages
* Create BE
* Upgrade
* Activate & boot
This does indeed make bad noises at the reboot time, though I think
they are not related to the zones, but to the fact that I have a
separate /var (normally I don't do this on my small instances, though I
always would on serious machines, for some reason I set it to have one
when I built the machine just now). Still, that's not great. I guess
I'll have to test it for patching with a separate /var now.
Single sparse root zone, root dataset at:
rpool/ROOT/10u7ZFSa/zoneRoot/common
lucreate barfs with:
Creating clone for <rpool/ROOT/10u7ZFSa/zoneRoot/common@10u8> on
<rpool/ROOT/10u8/zoneRoot/common-10u8>.
cannot mount 'rpool/ROOT/10u8/zoneRoot/common-10u8': legacy mountpoint
use mount(1M) to mount this filesystem
ERROR: Failed to mount dataset <rpool/ROOT/10u8/zoneRoot/common-10u8>
legacy is not an absolute path.
WARNING: split filesystem file system type <zfs> cannot inherit
mount point options <-> from parent filesystem file
type <-> because the two file systems have different types.
--
Ian Collins
>On 2009-10-18 00:30:37 +0100, Ian Collins <ian-...@hotmail.com> said:
>> I'd be interested to see if you can live upgrade from update 7 to
>> update 8 on the system with ZFS root and zones.
>I'd not tried this, but I have now. Basic process (this is on an x86 system):
>* 10u7 install from media, ZFS root.
>* Make a zone (ZFS root fs for zone)
>* Remove 10u7 LU packages (SUNWlucfg, SUNWlur, SUNWluu, SUNWluzone),
>add 10u8 packages
>* Create BE
The LU packages are the same. It's the LU patches that you need.
--
-Gary Mills- -Unix Group- -Computer and Network Services-
> The LU packages are the same. It's the LU patches that you need.
No, adding the packages is enough. Or rather: adding newer packages
effectively adds the patch you need. (10u7 has 121431-37 on x86, but
after adding the packages from 10u8 you'll find you magically have
121431-43.
> Single sparse root zone, root dataset at:
>
> rpool/ROOT/10u7ZFSa/zoneRoot/common
>
> lucreate barfs with:
>
> Creating clone for <rpool/ROOT/10u7ZFSa/zoneRoot/common@10u8> on
> <rpool/ROOT/10u8/zoneRoot/common-10u8>.
> cannot mount 'rpool/ROOT/10u8/zoneRoot/common-10u8': legacy mountpoint
> use mount(1M) to mount this filesystem
> ERROR: Failed to mount dataset <rpool/ROOT/10u8/zoneRoot/common-10u8>
> legacy is not an absolute path.
> WARNING: split filesystem file system type <zfs> cannot inherit
> mount point options <-> from parent filesystem file
> type <-> because the two file systems have different types.
OK, that's interesting as its similar to what I see, but not the same,
and at a different stage in the process. I've run out of time now, but
I did another test with 10u7, single zone with it;s own ZFS (like
yours), but no separate /var and this works OK to do lucreate /
luactivate / reboot (didn't do the upgrade). So may be it is /var, or
something - somehow it looks like it's getting confused with mount
options though. Clearly whatever I was doing last week was not enough
to tickle it.
Not a good enough reason. My users want root too, and I say no, and the
place still functions just fine. Having to babysit/coach users on
individual filesystems for people is the last way I'd want to waste my
time.
I'd rather not get fired for having the business shut down in the middle
of the day by setting quotas on the oracle user to save $2 of disk space.
I'm no ZFS expert, but I drill any other admin I run into about whatever
quirks they've noticed about it, since there used to be lots of them.
Using it helps too.
> What would be nice is a BUNCH of individual "Here's how *I* did it, switching
> from ufs to zfs, switching my backup procedures -- and if I had to do
> it again, I'd do it this other way, and why.:
I migrated a large mail store from UFS to ZFS over a few months. The new
ZFS mirrors were setup, compression enabled, then tested with lots of R/W
activity for months then more and more mail was ported over.
Email from these days seems to compress about 1.18 times or something.
Nothing too exciting, but 18% of hundreds of gigs adds up.
Annoyances are the zfs share commands seem completely useless. I still NFS
share stuff the old way as there doesn't seem to be a way to do it with
long permissions lists with "zfs share".
I'm not sure what the deal with that is.
The ZFS module in webconsole is good to play with if you're learning ZFS.
ZFS snapshots are still pretty phoney compared to the real deal, like
NetApp has. Having to cd into some whole other filesystem instead of just
cd .snapshot from wherever you are just obnoxious to the point of even
being useless.
> I'd rather not get fired for having the business shut down in the middle
> of the day by setting quotas on the oracle user to save $2 of disk space.
I wasn't actually meaning that. I meant that fs-per-user is a good
idea, and is perfectly viable when you only have a tiny number of
users. For instance I can see quite significant advantages for being
able to snapshot at the user level, when that "user" is some role.
> > No, but they might want snapshots, for example. So you create a file
> > system, give the appropriate permission to the user, possibly set a quota
> > and leave him be.
>
> Not a good enough reason. My users want root too, and I say no, and the
> place still functions just fine. Having to babysit/coach users on
> individual filesystems for people is the last way I'd want to waste my
> time.
Well, it's not that they'll want snapshots if noone explains them what
they can do with them. I don't know who your users are, but with my users
I find it easier to delegate their FOOBAR recovery to them. I have to
write some simple instructions, but they are smart enough to help
themselves. Sometimes I have to say no and point to the already written
instructions, but not often. If you treat them like intelligent beings,
they start to act like intelligent beings. Mostly because they have no
alternative. :-)
--
.-. .-. Yes, I am an agent of Satan, but my duties are largely
(_ \ / _) ceremonial.
|
| da...@fly.srk.fer.hr
Did you add the update 8 LU packages before the lucreate?
--
Ian Collins
> Did you add the update 8 LU packages before the lucreate?
Yes, or I think I did. I'll try again later.
--
Ian Collins
Why use long permissions lists rather than ACLs on the filesystems?
> ZFS snapshots are still pretty phoney compared to the real deal, like
> NetApp has. Having to cd into some whole other filesystem instead of just
> cd .snapshot from wherever you are just obnoxious to the point of even
> being useless.
ZFS snapshots certainly are the real deal. Navigating directories is no
worse (and a lot quicker) than using ufsrestore. It's also another case
where multiple and/or nested filesystems help.
--
Ian Collins
A nice safe playground is exactly what I need, I guess.
Since this is the first I've heard of "webconsole", please say a bit
about how to learn some zfs via it.
Hopefully, much of this long thread might someday end up in a
"learn-ZFS-ins-and-outs" FAC, so if you or anyone else has
the time and interest, say a bit (or maybe a lot) more, expanding
the "is good to play with" in the top line.
THANKS!
David
Whatever might be more useful to those who later come
across this thread or thread-turned-into-threads, for learning
some ins and outs of zfs, zfs backups, nested zfs's, etc.
THANKS!]
In article <7k2jhbF...@mid.individual.net>,
As I suggested in my immediately prior response (to the post just
before yours), assuming that much of what's said here in this
thread will end up in a learn-zfs FAQ, please you and others
elaborate on that last paragraph above, on:
1: doing restores via navigating directories (meaning zfs
filesystems organized into directories? Or just plain
zfs-less directories? Or what?) is faster and easier
than restoring via ufsrestore (partitions, I guess, or
directories (within a partition) there too?).
2: (is another case where multiple filesystems help).
3: is another case where NESTED FILESYSTEMS help.
Somewhere much more has to be said about nested
zfs filesystems. Like:
a: just what is a snapshot of a zfs filesystem
that has other ones nested inside it.
b: suppose you make snapshots of EACH of those
filesystems -- father, son, grandson, etc.
What do you end up with, and what can you do with
them?
Can you or can you not dispose of one or more or all
of the lower ones?
c: Exactly what is/are the USES of NESTED zfs filesystems?
d: people here seem to be agreeing that the way to make
off-site eg tape backups is via one or another version
of TAR: sun-tar, gnu-tar, or STAR.
da: are each of those tars equally equipped for handling
zfs backups?
db: ditto for backing up NESTED zfs filesystems?
dc: have any of them (or any other good-for-backups tools)
have zfs-SPECIFIC features added?
de: do any of them NEED to have zfs-specific features added?
Looking at the above, maybe I should of posted these as separate
threads, since each could generate lots of useful talk.
I'll leave that up to you guys -- maybe you want to create (surely
there is a newsgroup-specific terminology for this) separately
named SUB-threads (ie of THIS thread).
[I'm trying to ask these ignoramous-based questions in a way
that all this wonderful information they elicit as responses --
that they can help not just me, but LOTS of people. (And no,
I am not trying to write a book -- am just trying to learn
enough to feel somewhat confident to use my new sunblade 2500 (red)
that's been sitting here for two months now, all set up,
ready to go (done MUCH thanks
to Dennis Clarke of Blastwave), me getting no real work done
therefore (time=money down the drain), because thus far I'm scared to
do much with it -- which, I admit, is pretty darned stupid,
but that's what I am, let's face it. :-( ]
(For me, the major fear point is how to do off-site backups --
thus the above questions about the various tars & zfs-features, etc.)
Anyway, maybe I can be of use by asking these questions, and you
patiently answering (and re-answering) them, and come up with
something useful for EVERYONE!
Again, maybe time to "tree" this thread?
David
It's easy enough to gain at the command line:
# mkfile 1g fakedisk1
# mkfile 1g fakedisk2
# zpool create testpool mirror `pwd`/fakedisk1 `pwd`/fakedisk2
# df -h /testpool
Filesystem size used avail capacity Mounted on
testpool 984M 21K 984M 1% /testpool
[ Play... ]
# zpool destroy testpool
You don't even need to go through the fuss of lofiadm to create dev nodes.
--
Brandon Hume - hume -> BOFH.Ca, http://WWW.BOFH.Ca/
Man, am I burning up the paper on my hp laserjet!
The wikipedia on zfs is pretty good -- best of all are all
the links at the bottom.
Likewise the best-practices thing -- ton of links there too,
to blogs, manuals, sites, etc.
Sure involves one HUGE amount of reading.
--------
Question: Sun's doc: who writes it?
Engineers (who know what they're talking about)?
(For sure, this ZFS is no simple thing!)
Or "professional technical-writers", who might not know
quite as much?
Comments?
FYI: for anything technical, at least, that wikipedia is just
amazing, no?
History, too. I guess for just about anything!
Maybe whoever started wikipedia should get a Noble (Nobel?) prize!
David
I'd guess that it's a collaboration! The technical writers have to
learn the subject matter from the engineers who can't write!!!!
Good technical writers are rare. Daniel D. McCracken ("Guide to Fortran
IV Programming") was one of the best.
The reason I asked -- or maybe not-so-subtlely implied -- is that,
even though I am just learning (well, trying to learn) about zfs,
I find *some* of the Sun zfs doc could benefit from
a technical editor and also a "make it read easier" editor.
Certainly zfs man-page could use some (or maybe more than some)
work.
Anyway, nothing we can do about it.
David
> The reason I asked -- or maybe not-so-subtlely implied -- is that,
> even though I am just learning (well, trying to learn) about zfs,
> I find *some* of the Sun zfs doc could benefit from
> a technical editor and also a "make it read easier" editor.
>
> Certainly zfs man-page could use some (or maybe more than some)
> work.
>
> Anyway, nothing we can do about it.
At least some of the man pages in OpenSolaris are "open source", so you
can raise bugs on them and try to get involved in fixing them.
--
Chris
Of course there is something you can do about it! At least if you are a
paying customer, and maybe even if you are not. You write a letter
pointing out the problems with the existing documentation and suggesting
improvements and send it to Sun Microsystems.
You need to be very specific. Cite the page and paragraph and point out
what's missing or incorrect.
Back in the early 1970's someone at Princeton University punched every
example in the IBM System/360 Utilities Manual on cards and submitted
the deck. The output was riddled with error messages! He sent the
output to IBM. It resulted in a complete rewrite of the manual.
Once upon a time, IBM's manuals had a page or two at the back of the
manual that you could write your comments on, fold, tape and mail. Most
other companies did something of the sort.
If a user here deletes something, they know there is a backup, and that it
make take hours to restore from tape.
It makes them very careful. We can go months without having to restore
files for users , which is a nice thing.
I personally find quotas extremely obnoxious relics from the past. If a
user needs space, let them use it. Every now and then it's time to remind
the biggest space hogs to clean things up, and that takes a few minutes to
take care of.
I still can't see why anybody would want thousands of filesystems.
Why not just map every file to a new filesystem next?
If you link to a doc on how this works, I'll read it.
>> ZFS snapshots are still pretty phoney compared to the real deal, like
>> NetApp has. Having to cd into some whole other filesystem instead of just
>> cd .snapshot from wherever you are just obnoxious to the point of even
>> being useless.
>
> ZFS snapshots certainly are the real deal. Navigating directories is no
> worse (and a lot quicker) than using ufsrestore. It's also another case
> where multiple and/or nested filesystems help.
ufsrestore and zfs snapshots are not even 3% as cool or easy to use as
what netapp has. I don't like commands that wrap around my terminal 4
times as may be needed with the paths needed to use ZFS snapshots.
man chmod.
--
Ian Collins
Boy, are you ever hooked on this snapshot thing. Netapp was just
following suite from DEC who used it with there flavor or unix. It was
implemented in there ADVFS filesystem snapshot utility. Of couse,
NetAPP is now famous for this technology....yah. For some reason....
yah.
Take her easy.
> ufsrestore and zfs snapshots are not even 3% as cool or easy to use as
> what netapp has. I don't like commands that wrap around my terminal 4
> times as may be needed with the paths needed to use ZFS snapshots.
Of course, if you actually cared about this you'd just write some
rather small scripts to avoid this issue.
Any idea what netapp costs for 1, maybe 2 systems?
(Yes, I can call them, but thought maybe you might know straigh off.)
Thanks!
David
Last install of them I did was pretty large, so the price was probably in
the high hundreds of thousands.
You pretty much get what you pay for with hardware and software like this.
There's no time wasted making "small scripts" to actually use the features
of the product itself.