Bug#678696: Event based block device handling (fixes USB and nested devices problem)

[Goswin von Brederlow]

Package: initramfs-tools
Version: 0.106
Severity: wishlist
Tags: patch

Hi,

the attached patch adds an event based loop for block devices to the
init script. New blockdevices are recorded in
/run/initramfs/block-events by an udev rule as they appear. The init
script repeadately waits for that and then calls
/scripts/local-block/* with a list of new devices storedin NEWDEVS
until $ROOT and $resume (if set) exists or a timeout is reached.

This fixes the problem that USB devices take too long to be
discovered and crypto, raid, lvm or multipath can't be started on
them. It also adds support for arbitrary nestings of them, e.g. raid5
over raid1.

For the event loop I needed the $resume device translated to the
device node earlier. This was previously done in
scripts/local-premount/resume. It was essentially the same code used
to translate the $ROOT so I created a new function translate_device()
in script/functions and used that for both $ROOT and $resume.

Note: hooks/block-event and scripts/block-event are executable, not
represented in the patch.

MfG
Goswin

-- System Information:
Debian Release: wheezy/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=de_DE (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/dash

Versions of packages initramfs-tools depends on:
ii cpio 2.11-7
ii klibc-utils 2.0~rc3-1
ii module-init-tools 3.16-1
ii udev 175-3.1

Versions of packages initramfs-tools recommends:
ii busybox 1:1.19.3-5

Versions of packages initramfs-tools suggests:
ii bash-completion 1:1.99-3

-- no debconf information

[Goswin von Brederlow]

Hi,

I filed bugs for cryptsetup, lvm, mdadm and multipath-tools to support
this solution (i.e. provide scripts/local-block/ sniplets) [#678688
#678691 #678692 #678693].

If anyone wants to test this prior to those packages being fixed the
scriplets below can be used.

MfG
Goswin

==> scripts/local-block/cryptoroot <==
#!/bin/sh

PREREQ=""

prereqs()
{
echo "$PREREQ"
}

case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac

if [ -x /scripts/local-top/cryptoroot ]; then
exec /scripts/local-top/cryptoroot
fi

==> scripts/local-block/lvm2 <==
#!/bin/sh

PREREQ=""

prereqs()
{
echo "$PREREQ"
}

case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac

if [ -x /scripts/local-top/lvm2 ]; then
exec /scripts/local-top/lvm2
fi

==> scripts/local-block/mdadm <==
#!/bin/sh

PREREQ=""

prereqs()
{
echo "$PREREQ"
}

case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac

if [ -x /scripts/local-top/mdadm ]; then
exec /scripts/local-top/mdadm
fi




--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

[Michael Prokop]

Hi,

f'up to our recent discussion we had on IRC

* Goswin von Brederlow [Sat Jun 23, 2012 at 09:25:28PM +0200]:

> the attached patch adds an event based loop for block devices to the
> init script. New blockdevices are recorded in
> /run/initramfs/block-events by an udev rule as they appear. The init
> script repeadately waits for that and then calls
> /scripts/local-block/* with a list of new devices storedin NEWDEVS
> until $ROOT and $resume (if set) exists or a timeout is reached.

> This fixes the problem that USB devices take too long to be
> discovered and crypto, raid, lvm or multipath can't be started on
> them. It also adds support for arbitrary nestings of them, e.g. raid5
> over raid1.

[...]

First of all thanks again for the patch and your helpful feedback on
IRC.

I've tested your patch based on top of current i-t git master
(v0.116) with a setup like:

BOOT_IMAGE=/boot/vmlinuz-3.14-2-amd64 root=/dev/mapper/vg0-root ro quiet

but it sadly fails to work as intended (it boots but doesn't find
the block devices until the timeout is kicking in). I didn't
investigate closer yet, but AFAICS it seems to be related to the
fact that /dev/mapper/vg0-root doesn't exist at that time yet.

If you or someones else finds time to try and possibly further
investigate I'd very much welcome and appreciate that.

regards,
-mika-

[Goswin von Brederlow]

If it boots after the timeout then the device must exist. So either
the test for it is wrong or the device only gets created after the
timeout.

> If you or someones else finds time to try and possibly further
> investigate I'd very much welcome and appreciate that.
>
> regards,
> -mika-

Questions:

Does your system boot without the patch? If not then you also need the
lvm patch to provide the hook script that scans for a VG when new
block devices are detected.

Do you see any messages about vg0 being activated before the timeout
or after?

MfG
Goswin

[Goswin von Brederlow]

On Wed, Mar 04, 2015 at 09:30:24PM +0000, Ben Hutchings wrote:
> Thanks for your work on this bug. I ended up with a somewhat different
> implementation as I don't think it's necessary to duplicate the
> information that udev provides, and as we may now need to mount more
> than one filesystem. But I should have credited you in the changelog
> for prototyping this, and I forgot to do so.
>
> Ben.

The idea with the udev rule was to wait up to ROOTDELAY seconds
without a new device apearing before giving up. Now you wait ROOTDELAY
seconds in total, which then can depend on the number of devices. Add
a new disk and you have to increase the ROOTDELAY.

Also it was ment so that block scripts could specifically target the
new devices instead of having to scan all devices every time. For
example say you have a crypt device and you forgot the password. Now I
think you will be asked 30 times for the password before the initramfs
gives up.

[Ben Hutchings]

On Tue, 2015-03-10 at 12:47 +0100, Goswin von Brederlow wrote:
> On Wed, Mar 04, 2015 at 09:30:24PM +0000, Ben Hutchings wrote:
> > Thanks for your work on this bug. I ended up with a somewhat different
> > implementation as I don't think it's necessary to duplicate the
> > information that udev provides, and as we may now need to mount more
> > than one filesystem. But I should have credited you in the changelog
> > for prototyping this, and I forgot to do so.
> >
> > Ben.
>
> The idea with the udev rule was to wait up to ROOTDELAY seconds
> without a new device apearing before giving up. Now you wait ROOTDELAY
> seconds in total, which then can depend on the number of devices.

It's now max(rootdelay, 30) because the rootdelay kernel parameter
wasn't meant for this purpose at all.

> Add new disk and you have to increase the ROOTDELAY.

I hope not!

> Also it was ment so that block scripts could specifically target the
> new devices instead of having to scan all devices every time. For
> example say you have a crypt device and you forgot the password. Now I
> think you will be asked 30 times for the password before the initramfs
> gives up.

True, but so far as I could see you didn't send scripts that made use of
that. We could still implement that later, I think.

And now that we potentially have to mount /usr (and possibly other
filesystems), not just root and swap, lvm2's script needed to be told
which device we're looking for, not which devices appeared.

Ben.

--
Ben Hutchings
Any smoothly functioning technology is indistinguishable from a rigged demo.

[Goswin von Brederlow]

On Wed, Mar 11, 2015 at 06:09:50PM +0000, Ben Hutchings wrote:
> On Tue, 2015-03-10 at 12:47 +0100, Goswin von Brederlow wrote:
> > On Wed, Mar 04, 2015 at 09:30:24PM +0000, Ben Hutchings wrote:
> > > Thanks for your work on this bug. I ended up with a somewhat different
> > > implementation as I don't think it's necessary to duplicate the
> > > information that udev provides, and as we may now need to mount more
> > > than one filesystem. But I should have credited you in the changelog
> > > for prototyping this, and I forgot to do so.
> > >
> > > Ben.
> >
> > The idea with the udev rule was to wait up to ROOTDELAY seconds
> > without a new device apearing before giving up. Now you wait ROOTDELAY
> > seconds in total, which then can depend on the number of devices.
>
> It's now max(rootdelay, 30) because the rootdelay kernel parameter
> wasn't meant for this purpose at all.
>
> > Add new disk and you have to increase the ROOTDELAY.
>
> I hope not!

On one system the PSU isn't big enough to spin up all disks at once.
So the SCSI controler is set to not start them on power on. Instead
they come up sequentially. So one disk takes 5s, 2 disks 10s, 3 disks
15s, ... accordingly you have to set the delay. Add another disk and
the total time goes up.

> > Also it was ment so that block scripts could specifically target the
> > new devices instead of having to scan all devices every time. For
> > example say you have a crypt device and you forgot the password. Now I
> > think you will be asked 30 times for the password before the initramfs
> > gives up.
>
> True, but so far as I could see you didn't send scripts that made use of
> that. We could still implement that later, I think.
>
> And now that we potentially have to mount /usr (and possibly other
> filesystems), not just root and swap, lvm2's script needed to be told
> which device we're looking for, not which devices appeared.
>
> Ben.

That isn't realy new. Debian already had root and swap. Adding a third
doesn't realy change anything. LVM should already have known what
devices to activate for root and swap.

The problem I see there is detecting what devices to bring up. The
/usr filesystem might be in a ZPOOL that uses a crypt device on a LVM
logical volume on a raid. Or any other complex nesting. Could be any
number of devices that are needed for /usr to become available. Again
nothing new for /usr, / and swap already have that problem.

Note: LVM has persistent metadata that tell it wether to bring up a
device or not. So I'm not sure it makes much sense to guess what
devices are needed and limiting lvm to only start those. The metadata
already has this functionality and is more reliable than guessing what
devices may be needed.