synchronous directory updates vs. soft updates

[Matthias Andree]

Hi,

I just took the time to re-read the UMich report on soft updates[1] and
the journaling vs. soft updates USENIX 2000 paper[2].

As I understand these, soft updates are much like async with the
exception that the on-disk writes track dependencies so as to maintain
file system integrity[*].

On Linux, Postfix employs chattr +S on ext2 file systems to enforce
synchronous directory updates, to be on the safe side.

But what about a BSD FFS with soft updates? Soft updates guarantee
integrity, but not persistence (i. e. data including file names is
guaranteed to persist once the system call has returned)[1 (subsection
5.2)][2 (section 6)].

So, AFAICS, when running Postfix on BSD, the file system needs to be
mounted "sync" (slow) or "noasync" (default), but not "soft
updates"/"softdep" or "async".

What is the canonical way on (Free)BSD with soft updates to guarantee
file name persistence (after rename()/link()) or ensure that unlink()ed
files don't reappear after a crash? On Linux, the current and canonical
way to achieve this is "open the parent directory and fsync it". Does
this approach work for BSD with soft updates?

Notes

[*] Other than reclaiming free space still marked as allocated.

References

[1] Gregory R. Ganger, Yale N. Patt, "Soft Updates: A Solution to the
Metadata Update Problem in File Systems",
http://www.ece.cmu.edu/~ganger/papers/CSE-TR-254-95/

[2] Margo I. Seltzer, Gregory R. Ganger, M. Kirk McKusick, Keith
A. Smith, Craig A. N. Soules, Christopher A. Stein, "Journaling
Versus Soft Updates: Asynchronous Meta-data Protection in File
Systems",
http://www.usenix.org/publications/library/proceedings/usenix2000/general/seltzer.html

[ ] M. Kirk McKusick, Gregory R. Ganger, "Soft Updates, A Technique for
Eliminating Most Synchronous Writes in the Fast Filesystem",
http://www.usenix.org/publications/library/proceedings/usenix99/full_papers/mckusick/mckusick.pdf

--
Matthias Andree
-
To unsubscribe, send mail to majo...@postfix.org with content
(not subject): unsubscribe postfix-users

[Wietse Venema]

Quoting from the McKusic+Ganger 1999 USENIX paper for the BSD
implementation of soft updates:

An important filesystem interface is accessed through the
``fsync'' system call. This system call requests that the
specified file be written to stable storage and that the system
call not return until all its associated writes have completed.

The prototype soft update implementation did not implement the
``fsync'' system call. The task of completing an ``fsync''
requires more than simply writing all the file's dirty data
blocks to disk. It also requires that any unwritten directory
entries that reference the file also be written, as well as
any unwritten directories between the file and the root of
the filesystem.

Doing fsync() right in FFS took a bit of work.

Wietse

Matthias Andree:

[Craig Sanders]

On Fri, Aug 03, 2001 at 11:59:44PM +0200, Matthias Andree wrote:

> On Linux, the current and canonical way to achieve this is "open the
> parent directory and fsync it". Does this approach work for BSD with
> soft updates?

SGI's journalling filesystem XFS has a mount option called osyncisdsync.
i haven't researched it in detail but i'm curious to know what
implications it might have for a mail server. is this similar to
soft-updates?

from the mount man page:

osyncisdsync
Make writes to files opened with the O_SYNC flag
set behave as if the O_DSYNC flag had been used
instead. This can result in better performance
without compromising data safety. However if this
option is in effect, timestamp updates from O_SYNC
writes can be lost if the system crashes.

btw, i've already benchmarked reiserfs vs XFS (on the same hardware)
and found that XFS is a LOT slower than reiserfs for the typical mail
server lots-of-little-files usage pattern. OTOH, XFS is still plenty
fast enough and uses significantly less CPU.

Version 1.01b ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
xfs 1G 13019 99 65285 34 16726 12 9041 66 33448 9 562.1 1
xfs logbufs=8 1G 13031 99 66354 34 16504 12 9008 66 33645 10 562.0 2
xfs lb=8 osyncis 1G 13122 99 66346 36 16292 11 8981 66 33306 10 564.5 2
reiserfs 1G 12287 99 63942 63 18234 14 10058 75 44877 15 559.4 2
reiserfs notail 1G 12305 99 63196 62 17312 13 10432 78 42547 14 554.5 2
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
xfs 16 4075 58 +++++ +++ 3608 50 4017 63 +++++ +++ 2815 45
xfs logbufs=8 16 4601 82 +++++ +++ 4000 65 4495 82 +++++ +++ 3051 56
xfs lb=8 osyncis 16 4292 79 +++++ +++ 4591 68 3357 59 +++++ +++ 3353 58
reiserfs 16 14588 99 +++++ +++ 16403 100 13459 98 +++++ +++ 13398 99
reiserfs notail 16 14447 99 +++++ +++ 15461 94 13823 100 +++++ +++ 13520 100

this is on a dual p3-933 with 512MB RAM and 500GB of SCSI U160 RAID5
disk (128MB non-volatile cache ram) that i built last week. it's for
a file server not a mail server, and i'll be using XFS on it (quota
support and NFS compatibility are essential for this box).

craig

--
craig sanders <c...@taz.net.au>

Fabricati Diem, PVNC.
-- motto of the Ankh-Morpork City Watch

[Wietse Venema]

Matthias Andree:

> wie...@porcupine.org (Wietse Venema) writes:
>
> > Quoting from the McKusic+Ganger 1999 USENIX paper for the BSD
> > implementation of soft updates:
> >

> > The prototype soft update implementation did not implement the
> > ``fsync'' system call. The task of completing an ``fsync''
> > requires more than simply writing all the file's dirty data
> > blocks to disk. It also requires that any unwritten directory
> > entries that reference the file also be written, as well as
> > any unwritten directories between the file and the root of
> > the filesystem.
>

> Well, that's only half of the story. It will catch open, link and
> rename, but how do you fsync() a symlink or, more Postfix specific, an
> unlink()?

Please, let's keep it to things relevant for Postfix queues.
symlinks aren't.

I don't care about unlink. If a file is deleted and it comes back,
the worst that can happen is that mail is delivered twice.

Wietse

[Wietse Venema]

High time for me to study the alternative Postfix queue that
uses append/truncate with a pool of existing files, instead
of creating/renaming/deleting files. It was a lot faster.