UFS logging on SDS mirrored filesystems?
Mario Pranjic
Can you tell me your experiences with UFS logging on filesystems
(including rootfs) that are mirrored with Sun DiskSuite on Solaris 9?
Is it worth it? Are there any drawbacks?
Thanks!
Mario.
Thomas Glanzmann
> Can you tell me your experiences with UFS logging on filesystems
> (including rootfs) that are mirrored with Sun DiskSuite on Solaris 9?
> Is it worth it? Are there any drawbacks?
Don't use logging and quota on the same filesystem. Everything else
should work and turning logging on has only benefits, from my point of
view.
Thomas
Tony Walton
I don't quite understand your question. A UFS filesystem is a UFS
filesystem, whether it lives on a mirrored volume or not. Why would
there be any difference?
--
Tony
Dragan Cvetkovic
I suspect that Mario is/was asking about bugs and/or performance problems
...
Bye, Dragan
--
Dragan Cvetkovic,
To be or not to be is true. G. Boole No it isn't. L. E. J. Brouwer
!!! Sender/From address is bogus. Use reply-to one !!!
Peter C. Tribble
Thomas Glanzmann <sith...@stud.uni-erlangen.de> writes:
>
> Don't use logging and quota on the same filesystem.
Why on earth not?
--
-Peter Tribble
MRC Rosalind Franklin Centre for Genomics Research
http://www.rfcgr.mrc.ac.uk/~ptribble/
Thomas Glanzmann
>> Don't use logging and quota on the same filesystem.
> Why on earth not?
because it doesn't work. Google a bit.
Thomas
Peter C. Tribble
Thomas Glanzmann <sith...@stud.uni-erlangen.de> writes:
> Hi,
>
>>> Don't use logging and quota on the same filesystem.
>
>> Why on earth not?
>
> because it doesn't work.
Really? We've had it in production since 1996 at least (probably a
trans device in 1996, regular ufs logging since about 1999). What
doesn't work? (I do remember a bug back in 1996.)
Thomas Glanzmann
> Is it worth it? Are there any drawbacks?
logging and rootfs ... there were some issues. But ask experienced
people. Something with 'first stage boot loader can't replay the log'?!
Thomas
Thomas Glanzmann
> Really? We've had it in production since 1996 at least (probably a
> trans device in 1996, regular ufs logging since about 1999). What
> doesn't work? (I do remember a bug back in 1996.)
UFS + LOGGING + Quota.
Thomas
Roland Mainz
Yeah, and more worse: Dumb, unpriviledged users can use UFS logging on
the root volume to corrupt a filesystem in a way that even fsck cannot
help anymore. The same "trick" could be used for sabotage as well since
it usually will not be discovered until the machine reboots. And at that
point it's too late since the system cannot be recovered... ;-(
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland...@nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)
Akop Pogosian
Please provide a URL that supports your claim. I have been using UFS
logging with quotas for years and it does work.
-akop
Akop Pogosian
I have done it on Solaris 8 and 9 and it seems to work fine. If you're
using OS older than Solaris 9, then make sure to install the latest
Disksuite patch which is NOT part of the Recommended patch cluster. It
does fix some important bugs.
-akop
Akop Pogosian
Opps, actually, you were right, I just saw that thread. However, it
looks like it doesn't work because some recent BUG and not because Sun
doesn't support this functionality. I have seen it work just fine on
Solaris 7,8, and 9. I'll make sure not to install the patch cluster on
Solaris 9 machines until they fix this problem.
-akop
Thomas Glanzmann
> Opps, actually, you were right, I just saw that thread. However, it
> looks like it doesn't work because some recent BUG and not because Sun
> doesn't support this functionality. I have seen it work just fine on
> Solaris 7,8, and 9. I'll make sure not to install the patch cluster on
> Solaris 9 machines until they fix this problem.
this just one bug. There is another one. Did you never had users that
complained that they're over quota, but wasn't? Quota accounts more
space than there is actually in use. How many users do you have on your
fileserver using quota and logging?
Thomas
Akop Pogosian
A few hundred users. Yes, I remember someone complaining about random
quota problems though that happened very rarely and most often because
of some sysadmin mistakes (like forgetting to start rpc.quotad after
enabling quotas of NFS servers) I can't remember of anyone complaining
about quotas any time recently.
-akop
Akop Pogosian
> Thomas Glanzmann wrote:
>> > Is it worth it? Are there any drawbacks?
>>
>> logging and rootfs ... there were some issues. But ask experienced
>> people. Something with 'first stage boot loader can't replay the log'?!
> Yeah, and more worse: Dumb, unpriviledged users can use UFS logging on
> the root volume to corrupt a filesystem in a way that even fsck cannot
> help anymore. The same "trick" could be used for sabotage as well since
> it usually will not be discovered until the machine reboots. And at that
> point it's too late since the system cannot be recovered... ;-(
Can you elaborate more on this problem? What's the trick? I have never
heard of it before. It certainly sounds like a sort of security
problem that Sun should have started working on as soon as they found
out about it.
-akop
Thomas Glanzmann
> Can you elaborate more on this problem? What's the trick? I have never
> heard of it before. It certainly sounds like a sort of security
> problem that Sun should have started working on as soon as they found
> out about it.
users report that they're over quota, but they have no files in there
home. You start a find over the filesystem and can verify this
information. After this you reboot your machine. Force an fsck, which
returns nothing and run quotacheck -av (when you don't force the fs
check quotacheck -av doesn't cleanup the users usage). This happens at
least once a week.
Thomas
Roland Mainz
The issue is quite old, around the days when Solaris 2.7 was FCS... and
"yes", I have reported that issue long ago...
Steps to reproduce:
The root filesystem must be mounted with logging enabled. Boot the
machine, enable all services (database etc., whatever you want). Then
suspend the machine to disk (this is easy, simply start the dtsession
screen lock, then hit the "Power" key. The power deamon wants to show
the dialog suspend/shutdown/cancel but cannot grab the Xserver lock
(because the screensaver is holding it) and therefore goes immediately
into suspend action (yes, I have reported that issue, too. Long ago. And
even quoted it in my old Solaris CDE buglist))).
If not all transactions have been written to disk the machine cannot
resume so hitting the "power" key will result in a error message. Then
type "boot" in OBP to boot the machine normally. Enable all services and
let the box power-up all services again.
Everything works nice... perfectly... until someone reboots the machine.
Then suddenly the machine starts the "resume-from-disk" sequence using
the kernel image stored on disk - using old kernel data. Maybe they are
weeks old, maybe months. Or even older. Older is "better" because it
means "more devastation" ... =:-)
The "old" data in the kernel image from disk will eat the filesystem for
breakfast (you loose more or less anything between the last reboot and
now).
You can write a small script which can do that "job" automatically,
placing those "mines" in a whole server farm. The next admin in charge
who tries to reboot the machines will have a problem. A big one... ;-(
Akop Pogosian
Oh, I see. This sounds viable but I have never run into this since we
prefer not to allow the users to suspend any workstations.
-akop
Mario Pranjic
wrote:
>I suspect that Mario is/was asking about bugs and/or performance problems
Yes, that is right. :)
I'm interested in your experience because nobody would like fs
corruption on production machies during combination of logging and
SDS. Maybe that is not the issue here, but I ask anyway since I don't
have experience with UFS logging on heavily loaded machines.
Mario.
Mario Pranjic
<akopps...@ocf.berkeley.edu.remuvthis.com> wrote:
>I have done it on Solaris 8 and 9 and it seems to work fine. If you're
>using OS older than Solaris 9, then make sure to install the latest
>Disksuite patch which is NOT part of the Recommended patch cluster. It
>does fix some important bugs.
I have Solaris 9 machines and a few Solaris 8, but I'm interested in
UFS logging on Solaris 9.
But, after what is told here, I'm not sure I really want to enable
this. :)
On some other OS-es like AIX, journaling is default behaviour, on all
filesystems and it works very good. Never had any problems with jfs.
I see that on Solaris there are some issues and I will have to think
twice before I enable UFS logging.
On Sun website they say only good things about UFS logging, but this
was expectable. ;)
Thanks for sharing your experience with logging. I will certanly have
it in mind.
Mario.
Peter C. Tribble
Thomas Glanzmann <sith...@stud.uni-erlangen.de> writes:
> Hi,
>
>> Opps, actually, you were right, I just saw that thread. However, it
>> looks like it doesn't work because some recent BUG and not because Sun
>> doesn't support this functionality. I have seen it work just fine on
>> Solaris 7,8, and 9. I'll make sure not to install the patch cluster on
>> Solaris 9 machines until they fix this problem.
>
> this just one bug. There is another one. Did you never had users that
> complained that they're over quota, but wasn't?
Nope.
> Quota accounts more
> space than there is actually in use.
Nope.
> How many users do you have on your
> fileserver using quota and logging?
Only about 20,000.
Thomas Glanzmann
than you're lucky. Or you give your users so much quota that they didn't
notice the problems.
Honestly,
Thomas
Gavin Maltby
>>this just one bug. There is another one. Did you never had users that
>>complained that they're over quota, but wasn't? Quota accounts more
>>space than there is actually in use. How many users do you have on your
>>fileserver using quota and logging?
>
>
> A few hundred users. Yes, I remember someone complaining about random
> quota problems though that happened very rarely and most often because
> of some sysadmin mistakes (like forgetting to start rpc.quotad after
> enabling quotas of NFS servers) I can't remember of anyone complaining
> about quotas any time recently.
As I understand it quotas were not designed in to UFS from the start
and are therefore a retrofitted feature. Without first-class
support from the filesystem they're difficult to do and so bugs
are more likely. As far as I knew the current UFS quota picture
is not that bad (given the preceding). ZFS will do quotas and many other
things trivially.
Gavin
Thomas Glanzmann
> ZFS will do quotas and many other things trivially.
we will test it as soon as we get access to it. :-) Should be available
with the next beta, I think.
Thomas
UNIX admin
> I'm interested in your experience because nobody would like fs
> corruption on production machies during combination of logging and
> SDS. Maybe that is not the issue here, but I ask anyway since I don't
> have experience with UFS logging on heavily loaded machines.
As far as rootfs, you can forget using logging. The ufs boot loader does
not currently know what to do with the FS meta log.
So, stari moj, you're busted as far as logging rootfs.
With other FileSystems you will actually have a performance gain when using
logging, including metadevices used under SVM / DiskSuite.
I've read somewhere that Solaris10 will finally have a ufs boot loader which
will understand the meta log, but that question is better answered by
Solaris engineers here than myself.
UNIX admin
> UFS logging on Solaris 9.
>
> But, after what is told here, I'm not sure I really want to enable
> this. :)
Believe me, you *want* to enable it on all FileSystems except for rootfs.
> I see that on Solaris there are some issues and I will have to think
> twice before I enable UFS logging.
No you don't. It works and it works really good. What has been described
here are very specific cases that happen under very specific conditions.
If you're using quota, then tell those bozos at FER / VIP to fork out the
dough and get bigger disks. If the govt. has enough dough to sign an
enterprise license with M$, then they have enough dough for bigger disk
arrays as well.
> On Sun website they say only good things about UFS logging, but this
> was expectable. ;)
It *is* good.
Akop Pogosian
>> I have Solaris 9 machines and a few Solaris 8, but I'm interested in
>> UFS logging on Solaris 9.
>>
>> But, after what is told here, I'm not sure I really want to enable
>> this. :)
> Believe me, you *want* to enable it on all FileSystems except for rootfs.
Is there a specific reason not to enable logging for / ? BTW, starting
with Solaris 10 logging will be enabled by default on all UFS file
systems, root or not.
--
Akop Pogosian
This space has been accidentally left blank.
Beardy
> UNIX admin <tripi...@hotmail.com> wrote:
>
>>>I have Solaris 9 machines and a few Solaris 8, but I'm interested in
>>>UFS logging on Solaris 9.
>>>
>>>But, after what is told here, I'm not sure I really want to enable
>>>this. :)
>
>
>>Believe me, you *want* to enable it on all FileSystems except for rootfs.
>
>
> Is there a specific reason not to enable logging for / ? BTW, starting
> with Solaris 10 logging will be enabled by default on all UFS file
> systems, root or not.
>
Certainly in S10 beta and Express, logging is on by default for the root
filesystem. The question posed by UNIX admin was whether the ufsboot
loader could cope with rolling forward metadata logs for the filesystem.
I have been bitten by log area corruption previously (requiring a full
fsck), so am keen also to know what receovery processes have been
incorporated.
Beardy.
Scott Howard
> Is there a specific reason not to enable logging for / ? BTW, starting
No, not really.
There are a few really, really edge cases where it can cause problems,
but these are really only if you go asking for trouble. (Things like
applying a patch and then panicing your machine)
> with Solaris 10 logging will be enabled by default on all UFS file
> systems, root or not.
Forget Solaris 10, it's on by default in Solaris 9 9/04!
From the release notes at http://docs.sun.com/db/doc/817-5770 :
-------------------------------------------------
UFS Logging Enabled by Default
This feature is new in the Solaris 9 9/04 release.
Logging is now enabled by default for all UFS file systems except under the following conditions:
* When logging is explicitly disabled
* If insufficient file system space exists for the log
In previous Solaris releases, you had to enable UFS logging manually.
UFS logging packages into a transaction the multiple metadata changes
that compose a complete UFS operation. Sets of transactions are recorded
in an on-disk log, and then applied to the actual UFS file system's
metadata.
UFS logging provides two advantages:
* If the file system is already consistent because of the transaction
log, you might not have to run the fsck command after a system crash or
an unclean shutdown.
* Starting in the Solaris 9 12/02 release, the performance of UFS
logging improves or exceeds the level of performance of nonlogging file
systems. This improvement can occur because a file system with logging
enabled converts multiple updates to the same data into single updates.
This capability reduces the number of overhead disk operations that are
required.
-------------------------------------------------
Scott
Charlie Giannetto
Peter C. Tribble <ptri...@hgmp.mrc.ac.uk> wrote:
>In article <slrnchiaul....@faui00u.informatik.uni-erlangen.de>,
> Thomas Glanzmann <sith...@stud.uni-erlangen.de> writes:
>> Hi,
>>
>>> Opps, actually, you were right, I just saw that thread. However, it
>>> looks like it doesn't work because some recent BUG and not because Sun
>>> doesn't support this functionality. I have seen it work just fine on
>>> Solaris 7,8, and 9. I'll make sure not to install the patch cluster on
>>> Solaris 9 machines until they fix this problem.
>>
>> this just one bug. There is another one. Did you never had users that
>> complained that they're over quota, but wasn't?
>
>Nope.
>
>> Quota accounts more
>> space than there is actually in use.
>
>Nope.
>
>> How many users do you have on your
>> fileserver using quota and logging?
>
>Only about 20,000.
We've got over 24,000 users and can pretty much mirror Peter's comments.
There was an issue that caused a crash if you tried to umount a filesystem
with logging and quotas enabled but that was fixed a while back. As for
other issues, we had to kick up the value of ndquot in /etc/system in order
to increase the quota table size but that's about it. Actually, that was
done several Solaris releases ago so I'm not entirely sure it necessary
anymore but we've stuck with it and have not run into any problems.
- Charlie
Andrew Gabriel
"UNIX admin" <tripi...@hotmail.com> writes:
>
>> I'm interested in your experience because nobody would like fs
>> corruption on production machies during combination of logging and
>> SDS. Maybe that is not the issue here, but I ask anyway since I don't
>> have experience with UFS logging on heavily loaded machines.
>
> As far as rootfs, you can forget using logging. The ufs boot loader does
> not currently know what to do with the FS meta log.
> So, stari moj, you're busted as far as logging rootfs.
For most users, this is a non-issue. It's pretty much only low
level Solaris developers who are routinely modifying the files
in the root filesystem which are read by the boot code before
the filesystem is mounted. Other than that, system keeling over
whilst applying patches to drivers and config files accessed
before root filesystem is mounted is the other small window.
I've run with logging root filesystems since Solaris 7, and I
do kernel development affecting these files, and I've not had
a problem. If you are paranoid, then after applying a patch, do
sync;sync # write data to disk
lockfs -f / # roll UFS log
> With other FileSystems you will actually have a performance gain when using
> logging, including metadevices used under SVM / DiskSuite.
Yes, and the gain gets drammatically better with more recent
releases.
> I've read somewhere that Solaris10 will finally have a ufs boot loader which
> will understand the meta log, but that question is better answered by
> Solaris engineers here than myself.
Yes, and I think the latest Solaris 9 update.
--
Andrew Gabriel
UNIX admin
> do kernel development affecting these files, and I've not had
> a problem. If you are paranoid, then after applying a patch, do
Actually so have I; however, I'm well aware of what you write of, whereas he
is (was) not.
UNIX admin
> with Solaris 10 logging will be enabled by default on all UFS file
> systems, root or not.
First, right now it has no effect. Second, as you've read here, there are
potential problems. I have "logging" as an option in all my /etc/vfstab
files for the /, but that's just me.
As for Solaris10, it'll kick ass, but until it gets past FCS, it's not an
option for me.
UNIX admin
>
> From the release notes at http://docs.sun.com/db/doc/817-5770 :
... Yes, but does `ufsboot` know what to do with that meta log for /?
That's the key detail here.
Juergen Keil
The following Solaris 9 patches look promising:
116548-03 SunOS 5.9: ufsboot Patch
114733-10 SunOS 5.9_x86: ufs and fsck Patch
Both patches include a fix for
4520944 ufsboot should properly handle a logging root filesystem
with non-clear log
Chris Thompson
Of course, for the really paranoid, there's an earlier problem: can the
15-sector bootblk written by installboot(1m) cope with the situation
when the metadata involved in finding /platform/[ARCH]/ufsboot is in
transition recorded only in the log?
Chris Thompson
Email: cet1 [at] cam.ac.uk
Beardy
>
> As for Solaris10, it'll kick ass, but until it gets past FCS, it's not an
> option for me.
>
>
It *does* kick ass, natch ;-) But I completely understand the current
supportability issues pre-FCS -- ie. there aren't (m)any. But waiting is
such torture ;-)
Beardy.