Scripts to start and stop ZFS correctly under Ubuntu Hardy

[Rudd-O]

Here they are. They do not support rootfs on ZFS.

root@karen: /etc #
cat /sbin/startzfs /sbin/stopzfs /etc/init.d/mountzfs.sh /etc/init.d/
umountzfs.sh

#!/bin/bash

set -e

for a in /dev/sd[abcdefg] ; do
/sbin/hdparm -q -W0 $a
done

ulimit -v unlimited

nice -n -15 /sbin/zfs-fuse "$@"
sleep 1

zfspid=`pidof zfs-fuse`

echo -17 > /proc/$zfspid/oom_adj

/sbin/zfs mount -a
-----------------
#!/bin/bash

set -e

/sbin/zfs unmount -a

sync
killall -TERM zfs-fuse

while true; do
set +e
pidof zfs-fuse > /dev/null
if [ "$?" == "1" ] ; then break ; fi
set -e
sleep 1
done

sync
----------------------
#! /bin/sh
### BEGIN INIT INFO
# Provides: mountzfs
# Required-Start: checkfs mountall
# Required-Stop:
# Should-Start: lvm
# Should-Stop:
# Default-Start: S
# Default-Stop:
# Short-Description: Turn ZFS-FUSE on and mount ZFS filesystems.
# Description: Start ZFS-FUSE subsystem
### END INIT INFO

PATH=/sbin:/bin
. /lib/init/vars.sh

. /lib/lsb/init-functions
. /lib/init/mount-functions.sh

if [ -r /etc/default/locale ]; then
. /etc/default/locale
export LANG
fi

do_start() {
#
# Mount local file systems in /etc/fstab.
#
pre_mountall
log_action_begin_msg "Activating ZFS-FUSE subsystem"
/sbin/startzfs
ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"
fi
post_mountall
}

case "$1" in
start|"")
do_start
;;
restart|reload|force-reload)
echo "Error: argument '$1' not supported" >&2
exit 3
;;
stop)
# No-op
;;
*)
echo "Usage: mountall.sh [start|stop]" >&2
exit 3
;;
esac

:
---------------------
#! /bin/sh
### BEGIN INIT INFO
# Provides: umountzfs
# Required-Start: umountnfs urandom
# Required-Stop:
# Default-Start: 0 6
# Default-Stop:
# Short-Description: Unmount ZFS filesystems and turn ZFS-FUSE off
safely.
# Description: Stop ZFS-FUSE subsystem
### END INIT INFO

PATH=/usr/sbin:/usr/bin:/sbin:/bin
. /lib/init/vars.sh

. /lib/lsb/init-functions

umask 022

do_stop () {
log_action_begin_msg "Stopping ZFS-FUSE subsystem"
/sbin/stopzfs
ES=$?
if [ "$ES" = 0 ]; then
log_success_msg "ZFS-FUSE subsystem stopped."
else
log_failure_msg "ZFS-FUSE subsystem failed to stop
with error code $ES."
fi
}

case "$1" in
start)
# No-op
;;
restart|reload|force-reload)
echo "Error: argument '$1' not supported" >&2
exit 3
;;
stop)
do_stop
;;
*)
echo "Usage: $0 start|stop" >&2
exit 3
;;
esac

:
--------------

You can then

update-rc.d mountzfs.sh start 36 S .
update-rc.d umountzfs.sh start 39 0 6 .

Now... can someone explain me what dark magic does the init.d
subsystem do in Ubuntu, that the umountfs/umountzfs script is invoked
with a stop argument when the /etc/rc6.d/ file starts with an S?

[Eric A]

On Tue, Jun 17, 2008 at 8:28 AM, Rudd-O <drago...@gmail.com> wrote:
> for a in /dev/sd[abcdefg] ; do
> /sbin/hdparm -q -W0 $a
> done

You don't need this anymore. Any time hdparm or sdparm would be able
to disable the write cache on a drive, zfs-fuse will flush the write
cache when necessary. Technically, there may be some very old drives
that only support enabling/disabling the write cache and not flushing
it, but that's like Pentium (1) era technology.

Cheers,
Eric

[Rudd-O]

I don't understand, this is contradictory to me. On one post, you're
saying that write barriers / fsync() don't work. On other place, i
read that ZFS achieves write atomicity by shoving two barriers: one
before uberblock write, and one after it. Somewhere else I read that
this does NOT WORK on Linux -- and the fsync() example that I wrote is
good proof of that. Finally, you say that hdparm is no longer needed.


So, how does zfs-fuse flush the write cache, and guarantee correct
ordering in writes nowadays?

On Jun 17, 8:11 pm, "Eric A" <erp...@gmail.com> wrote:

[Eric A]

On Tue, Jun 17, 2008 at 9:22 PM, Rudd-O <drago...@gmail.com> wrote:
>
> I don't understand, this is contradictory to me. On one post, you're
> saying that write barriers / fsync() don't work. On other place, i
> read that ZFS achieves write atomicity by shoving two barriers: one
> before uberblock write, and one after it. Somewhere else I read that
> this does NOT WORK on Linux -- and the fsync() example that I wrote is
> good proof of that. Finally, you say that hdparm is no longer needed.
>
>
> So, how does zfs-fuse flush the write cache, and guarantee correct
> ordering in writes nowadays?

Write barriers work most of the time, but they are a kernel
abstraction rather than a feature of the hardware. zfs-fuse does not
use them. fsync() doesn't work on Linux. zfs-fuse gets around these
problems by opening the device files using O_DIRECT and flushing the
write cache at the appropriate time by talking directly to the
hardware.

Cheers,
Eric

[Rudd-O]

OK, thanks for the info.

Here are sum new initscripts that hopefully will work:

-----------------------------------------
root@karen: ~ #
cat /etc/init.d/zfs

#! /bin/sh
### BEGIN INIT INFO

# Provides: zfs
# Required-Start: mountall
# Required-Stop: sendsigs
# Should-Start:
# Should-Stop:
# Default-Start:
# Default-Stop:
# Short-Description: Enable/disable the ZFS-FUSE subsystem
# Description: Control ZFS-FUSE subsystem
### END INIT INFO

PIDFILE=/var/run/zfs-fuse.pid
LOCKFILE=/var/lock/zfs/zfs_lock
PATH=/sbin:/bin:/usr/bin

. /lib/init/vars.sh

. /lib/lsb/init-functions
. /lib/init/mount-functions.sh

if [ -r /etc/default/locale ]; then
. /etc/default/locale
export LANG
fi

do_start() {

test -x /sbin/zfs-fuse || exit 0
PID=`cat "$PIDFILE" 2> /dev/null`
if [ "$PID" != "" ]
then
if kill -0 $PID
then
echo "ZFS-FUSE is already running"
exit 3
else
# pid file is stale, we clean up shit
log_action_begin_msg "Cleaning up stale ZFS-
FUSE PID files"
rm -f /var/run/sendsigs.omit.d/zfs-fuse
"$PIDFILE"
log_action_end_msg 0
fi
fi

pre_mountall

log_action_begin_msg "Starting ZFS-FUSE process"
ulimit -v unlimited
zfs-fuse -p "$PIDFILE"

ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"

post_mountall
exit 3
fi

for a in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
do
PID=`cat "$PIDFILE"`
[ "$PID" != "" ] && break
sleep 1
done

if [ "$PID" = "" ]
then
log_action_end_msg 1 "ZFS-FUSE did not start or create
$PIDFILE"
post_mountall
exit 3
else
log_action_end_msg 0
fi

log_action_begin_msg "Immunizing ZFS-FUSE against OOM kills
and sendsigs signals"
cp "$PIDFILE" /var/run/sendsigs.omit.d/zfs-fuse
echo -17 > "/proc/$PID/oom_adj"

ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"

post_mountall
exit 3
fi

log_action_begin_msg "Mounting ZFS filesystems"
zfs mount -a

ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"

post_mountall
exit 3
fi

log_action_begin_msg "Increasing ZFS-FUSE priority"
renice -15 -g $PID > /dev/null

ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"

post_mountall
exit 3
fi

post_mountall
}

do_stop () {
test -x /sbin/zfs-fuse || exit 0
PID=`cat "$PIDFILE" 2> /dev/null`
if [ "$PID" = "" ] ; then
# no pid file, we exit
exit 0
elif kill -0 $PID 2> /dev/null; then
# pid file and killable, we continue
true
else
# pid file is stale, we clean up shit
log_action_begin_msg "Cleaning up stale ZFS-FUSE PID
files"
rm -f /var/run/sendsigs.omit.d/zfs-fuse "$PIDFILE"
log_action_end_msg 0
exit 0
fi

pre_mountall

log_action_begin_msg "Syncing disks"
sync
log_action_end_msg 0

log_action_begin_msg "Unmounting ZFS filesystems"
zfs unmount -a

ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"

post_mountall
exit 3
fi

post_mountall # restore /var/lock and /var/run to their right
places

log_action_begin_msg "Terminating ZFS-FUSE process gracefully"
kill -TERM $PID

for a in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
do
kill -0 $PID 2> /dev/null
[ "$?" != "0" ] && break
sleep 1
done

if kill -0 $PID 2> /dev/null
then
log_action_end_msg 1 "ZFS-FUSE refused to die after 15
seconds"
post_mountall
exit 3
else
rm -f /var/run/sendsigs.omit.d/zfs-fuse "$PIDFILE"
log_action_end_msg 0
fi

log_action_begin_msg "Syncing disks again"
sync
log_action_end_msg 0
}

case "$1" in
start)
do_start
;;
stop)
do_stop
;;
status)
PID=`cat "$PIDFILE" 2> /dev/null`
if [ "$PID" = "" ] ; then
echo "ZFS-FUSE is not running"
exit 3
else
if kill -0 $PID
then
echo "ZFS-FUSE is running, pid $PID"
zpool status
exit 0
else
echo "ZFS-FUSE died, PID files stale"
exit 3
fi
fi

;;
restart|reload|force-reload)
echo "Error: argument '$1' not supported" >&2
exit 3
;;

*)
echo "Usage: $0 start|stop|status" >&2
exit 3
;;
esac

:
-----------------------------------------
root@karen: ~ #
cat /etc/init.d/mountzfs

#! /bin/sh
### BEGIN INIT INFO
# Provides: mountzfs

# Required-Start: mountall
# Required-Stop:

# Default-Start: S
# Default-Stop:

# Short-Description: Enable ZFS-FUSE.
# Description: Enable and mount ZFS filesystems.
### END INIT INFO

case "$1" in
start|stop|"")
/etc/init.d/zfs start
exit $?

;;
restart|reload|force-reload)
echo "Error: argument '$1' not supported" >&2
exit 3
;;

*)
echo "Usage: mountzfs [start|stop]" >&2
exit 3
;;
esac

:
-----------------------------------------
root@karen: ~ #
cat /etc/init.d/umountzfs

#! /bin/sh
### BEGIN INIT INFO
# Provides: umountzfs

# Required-Start: sendsigs
# Required-Stop:
# Default-Start: 6 0
# Default-Stop:
# Short-Description: Disable ZFS-FUSE.
# Description: Unmount and disable ZFS filesystems.
### END INIT INFO

case "$1" in
start|stop|"")
/etc/init.d/zfs stop
exit $?

;;
restart|reload|force-reload)
echo "Error: argument '$1' not supported" >&2
exit 3
;;

*)
echo "Usage: umountzfs [start|stop]" >&2
exit 3
;;
esac

:
-----------------------------------------
root@karen: ~ #
ls -l /etc/rc?.d/*zfs*
lrwxrwxrwx 1 root root 19 2008-06-18 03:52 /etc/rc0.d/S35umountzfs -
> ../init.d/umountzfs
lrwxrwxrwx 1 root root 19 2008-06-18 03:52 /etc/rc6.d/S35umountzfs -
> ../init.d/umountzfs
lrwxrwxrwx 1 root root 18 2008-06-18 03:52 /etc/rcS.d/S35mountzfs -
> ../init.d/mountzfs
-----------------------------------------
root@karen: ~ #

[Rudd-O]

And a new revision, because the old script hung like a horse when
mounting /usr:

root@karen: ~ #
cat /etc/init.d/zfs
#! /bin/sh
### BEGIN INIT INFO
# Provides: zfs
# Required-Start: mountall
# Required-Stop: sendsigs
# Should-Start:
# Should-Stop:
# Default-Start:
# Default-Stop:
# Short-Description: Enable/disable the ZFS-FUSE subsystem
# Description: Control ZFS-FUSE subsystem
### END INIT INFO

PIDFILE=/var/run/zfs-fuse.pid
LOCKFILE=/var/lock/zfs/zfs_lock

. /lib/init/vars.sh

. /lib/lsb/init-functions
. /lib/init/mount-functions.sh

export PATH=/sbin:/bin
unset LANG
ulimit -v unlimited

do_start() {
test -x /sbin/zfs-fuse || exit 0
PID=`cat "$PIDFILE" 2> /dev/null`
if [ "$PID" != "" ]
then

if kill -0 $PID 2> /dev/null

then
echo "ZFS-FUSE is already running"
exit 3
else
# pid file is stale, we clean up shit
log_action_begin_msg "Cleaning up stale ZFS-
FUSE PID files"
rm -f /var/run/sendsigs.omit.d/zfs-fuse
"$PIDFILE"
log_action_end_msg 0
fi
fi

pre_mountall

log_action_begin_msg "Starting ZFS-FUSE process"

mkdir -p /var/run/sendsigs.omit.d

cp "$PIDFILE" /var/run/sendsigs.omit.d/zfs-fuse
echo -17 > "/proc/$PID/oom_adj"
ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"
post_mountall
exit 3
fi

log_action_begin_msg "Mounting ZFS filesystems"

zfs mount -a
ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"
post_mountall
exit 3
fi

if [ -x /usr/bin/renice ] ; then
log_action_begin_msg "Increasing ZFS-FUSE priority"
/usr/bin/renice -15 -g $PID > /dev/null

ES_TO_REPORT=$?
if [ 0 = "$ES_TO_REPORT" ]
then
log_action_end_msg 0
else
log_action_end_msg 1 "code $ES_TO_REPORT"
post_mountall
exit 3
fi

true

[Rudd-O]

The sendsigs omit pid protocol is USELESS:

https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/87763

there are my comments. over here, killall5 hangs indefinitely, even
though I've confirmed the omit.d pid is correct.