"Waiting for root file system..." hang solved
Cameron L. Spitzer
asking, so this is for the next person with the same problem.
I installed a Debian Etch workstation, tasksel: desktop, guided
partitioning, the six partitions (/ /usr /var /home /tmp swap) way.
Works great. I upgraded to Lenny, the October 29 2007 weekly CD.
This leaves a GRUB menu with an Etch kernel and a Lenny kernel.
The Etch kernel boots just fine. The Lenny kernel hangs at
"Waiting for root file system..."
and eventually you get that (initramfs) prompt from the Busybox
shell, which means it never found the root FS.
Turns out this is because Etch thinks this PATA drive is
/dev/hda, but Lenny thinks the exact same drive is /dev/sda.
So the /etc/fstab and /boot/grub/menu.lst files have the
root file system as /dev/hda1, which no longer exists.
I hear ubuntu Fiesty has the same problem.
The fix is to edit /etc/fstab and /boot/grub/menu.lst and
change all the hard drive partition device names to unique file
system volume labels. /dev/hda1 becomes LABEL=a-root or whatever
you like. Then use e2label(8) to add volume labels to
all the file systems. I also labeled the swap image.
While you're at it, /etc/uswsusp.conf is wrong, too.
(Can that file use a swap volume label instead of a device name?
The manpage doesn't say.)
This causes a pause during boot, where resume: is looking for
the wrong swap device.
Once all the partitions are labeled, you can make new
initrd images. update-initramfs -u -k all
You could have a debate about whether this is an installer
bug, a kernel package bug, a udev bug, or operator error.
Cameron
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Johannes Wiedersich
Hash: SHA1
Cameron L. Spitzer wrote:
[snip upgrade instructions]
Thanks for posting your experience! I am sure it will be useful to others.
> You could have a debate about whether this is an installer
> bug, a kernel package bug, a udev bug, or operator error.
If I understand correctly, you upgraded the kernel and the new kernel
would not boot. Then it would be a kernel bug.
- From the installation/upgrade instructions from sarge to etch I
remember, that one was supposed to upgrade the kernel and just the
kernel, then reboot and upgrade other packages. Is this still the case?
Did you follow the upgrade instructions?
Johannes
NB: Unfortunately, I could not find any recent installation / upgrade
instructions for lenny. The development version [1] still claims to be
about the upgrade from sarge to etch and always refers to etch and never
to lenny [2].
[1] http://d-i.alioth.debian.org/manual/
[2] http://d-i.alioth.debian.org/manual/en.i386/install.en.pdf
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFHKui+C1NzPRl9qEURAngHAJ4zz3A0yAKkaa9WT0Qhj5nBvnCFGgCcDbQY
Fgr/+suvTpjAEmDs1WPYkK4=
=mHxd
-----END PGP SIGNATURE-----
c...@truffula.sj.ca.us
In article <9l0Hy...@gated-at.bofh.it>, Johannes Wiedersich wrote:
>
> Cameron L. Spitzer wrote:
> [snip upgrade instructions]
>
> Thanks for posting your experience! I am sure it will be useful to others.
>
>> You could have a debate about whether this is an installer
>> bug, a kernel package bug, a udev bug, or operator error.
>
> If I understand correctly, you upgraded the kernel and the new kernel
> would not boot. Then it would be a kernel bug.
That was my initial conclusion. But then I spent some time
googling for the error messages. A lot of people have had this
same hang, and most of them got there by some other path than I did.
So I think it may be a more general problem than that.
My friend in Los Angeles tried to install Ubuntu for a friend,
and got stuck "waiting for root file system" in the middle of
a fresh install from CD. When he booted his trusty Knoppix CD
it revealed the root file system was just fine. I suspect udev
device names are less persistent than we have assumed they are.
> - From the installation/upgrade instructions from sarge to etch I
> remember, that one was supposed to upgrade the kernel and just the
> kernel, then reboot and upgrade other packages. Is this still the case?
> Did you follow the upgrade instructions?
That's what I did. When I installed an Etch kernel on Sarge, it
pulled in a new libc, locales, and a few other things. It replaced
module-utils. I think it replaced devfs with udev.
When I installed a Lenny kernel on a fresh Etch,
it just put the new kernel in along side the old one. Nothing else
new.
I agree udev is an improvement over devfs or just having all
possible static device nodes. But would it be unreasonable to
create static device nodes for the devices in the boot path?
Or in /etc/fstab?
Can I do that, or will udev just take them away?
I've read the udev manpages and I just get more and more confused.
It's that old unix documentation canard that "examples will just
limit your creativity." Udev needs a top-down explanation and an
introductory tutorial with complete examples. There's a "view
from ten thousand feet" overview (the conference paper), and lots
of details (the manpages), but no bridge between them.
Cameron
Andrew Sackville-West
> [This message has also been posted to linux.debian.user.]
> In article <9l0Hy...@gated-at.bofh.it>, Johannes Wiedersich wrote:
> >
> > Cameron L. Spitzer wrote:
> > [snip upgrade instructions]
> >
> > Thanks for posting your experience! I am sure it will be useful to others.
> >
> >> You could have a debate about whether this is an installer
> >> bug, a kernel package bug, a udev bug, or operator error.
> >
> > If I understand correctly, you upgraded the kernel and the new kernel
> > would not boot. Then it would be a kernel bug.
>
> That was my initial conclusion. But then I spent some time
> googling for the error messages. A lot of people have had this
> same hang, and most of them got there by some other path than I did.
> So I think it may be a more general problem than that.
>
> My friend in Los Angeles tried to install Ubuntu for a friend,
> and got stuck "waiting for root file system" in the middle of
> a fresh install from CD. When he booted his trusty Knoppix CD
> it revealed the root file system was just fine. I suspect udev
> device names are less persistent than we have assumed they are.
yeah, this is probably *not* a kernel bug but more likely either a
udev bug or initramsf-tools bug. Something got changed there in the
device naming and that's not really the kernel's fault, so far as I
know.
BTW, were you able to boot through the busybox? I've had to learn my
way around that having just reconfigured my laptop. The critical item
is the contents of $ROOT. The value of $ROOT gets set by the kernel
command line and if it doesn't match, then you have trouble. If you
change that to the appropriate value you can then 'exit' busybox and
the boot will carry on. Once you're up and running, then rebuil the
initrd's.
A
Cameron L. Spitzer
>
> On Fri, Nov 02, 2007 at 03:33:06PM -0700, c...@truffula.sj.ca.us wrote:
>> In article <9l0Hy...@gated-at.bofh.it>, Johannes Wiedersich wrote:
>> > Cameron L. Spitzer wrote:
Lenny kernel wouldn't boot, but the Etch kernel still worked. ]
>> > If I understand correctly, you upgraded the kernel and the new kernel
>> > would not boot. Then it would be a kernel bug.
[cls:]
>> My friend in Los Angeles tried to install Ubuntu for a friend,
>> and got stuck "waiting for root file system" in the middle of
>> a fresh install from CD. When he booted his trusty Knoppix CD
>> it revealed the root file system was just fine. I suspect udev
>> device names are less persistent than we have assumed they are.
> yeah, this is probably *not* a kernel bug but more likely either a
> udev bug or initramsf-tools bug. Something got changed there in the
> device naming and that's not really the kernel's fault, so far as I
> know.=20
>
> BTW, were you able to boot through the busybox?
I didn't have to try. The Etch kernel+initrd still worked.
As soon as the system was up I changed /etc/fstab and grub/menu.lst
to use volume labels which make the Lenny kernel+initrd work too.
> I've had to learn my
> way around that having just reconfigured my laptop. The critical item
> is the contents of $ROOT. The value of $ROOT gets set by the kernel
> command line and if it doesn't match, then you have trouble. If you
> change that to the appropriate value you can then 'exit' busybox and
> the boot will carry on.
I didn't know that. If I hadn't had a working option in GRUB
I would have tried editing the kernel command line next.
I've also rescued Debian by booting Knoppix, mounting stuff,
and running a chroot shell.
> Once you're up and running, then rebuil the
> initrd's.
I actually tried that before going to volume labels. Rebuilding
the initrd puts the same old /etc/fstab in the new initrd image.
That doesn't get you past the udev hang. I guess a more sophisticated
update-initrd would alert you to the difference between the
current mtab and the /etc/fstab contents. It wouldn't know whether
the difference was intended, so it would have to ask what to do.
And if I'd put a new device-names fstab in the Etch initrd then
my Etch kernel wouldn't have worked any more.
Come to think of it, I only learned about volume labels a few
months ago, solving a similar problem. I installed an Etch web
server on /dev/hde (a drive on an add-in ATA interface card),
in a test machine that already had an Etch workstation on /dev/hda.
And when I launched the hde kernel with GRUB it booted the
workstation instead! I fiddled around with the udev rules
but they were so poorly documented I wasn't confident I could
upgrade the machine remotely.
I've been using Debian for a long time. It's just *weird* to
see anything broken like that.
Andrew Sackville-West
> In article <9ldY5...@gated-at.bofh.it>, Andrew Sackville-West wrote:
>
> > I've had to learn my
> > way around that having just reconfigured my laptop. The critical item
> > is the contents of $ROOT. The value of $ROOT gets set by the kernel
> > command line and if it doesn't match, then you have trouble. If you
> > change that to the appropriate value you can then 'exit' busybox and
> > the boot will carry on.
>
> I didn't know that. If I hadn't had a working option in GRUB
> I would have tried editing the kernel command line next.
> I've also rescued Debian by booting Knoppix, mounting stuff,
> and running a chroot shell.
it turns out you can do a reasonable amount of stuff in that busybox
shell and if the system is close to booting, you can get it to
go. What I was missing, and desperately wanted, was some kind of text
editor. I ended up have to do some funky 'cat'ing of files and text
input to tweak an encryption-key script. Not fun without an editor,
but you sure can do it...
>
>
> > Once you're up and running, then rebuil the
> > initrd's.
>
> I actually tried that before going to volume labels. Rebuilding
> the initrd puts the same old /etc/fstab in the new initrd image.
sorry, updating /etc/fstab apparently doesn't go without
saying... ;).
> That doesn't get you past the udev hang. I guess a more sophisticated
> update-initrd would alert you to the difference between the
> current mtab and the /etc/fstab contents.
There are a handful of issues I've seen with currnet initramsfs-tools
that probably need addressing. The one I ran across: the lvm and
encrypted root scripts expect /dev/mapper/ names with hyphens in
them. I have /dev/mapper/vgcrypt-root (an LVM volume on top of
encrypted partition). I prefer to access these things through
/dev/vgcrypt/root (or whatever it was), but the scripts are
expecting that hyphenated name... it appears to only be documented
in the code itself. oh well. it works now...
> I've been using Debian for a long time. It's just *weird* to
> see anything broken like that.
somewhere Linus talks about breaking UUID names as well, on purpose,
because he thinks there should be some other persistent naming method
and since UUID is generated in user space, its not reliable. I don't
know, but watch out for that.
A
Douglas A. Tutty
> it turns out you can do a reasonable amount of stuff in that busybox
> shell and if the system is close to booting, you can get it to
> go. What I was missing, and desperately wanted, was some kind of text
> editor. I ended up have to do some funky 'cat'ing of files and text
> input to tweak an encryption-key script. Not fun without an editor,
> but you sure can do it...
>
You mean that the busybox doesn't have any editor at all? The busybox
package (not necessarily the same as the initrd version) has vi and ed.
No ed in the initrd?
Doug.
Andrew Sackville-West
> On Sat, Nov 03, 2007 at 09:14:44PM -0700, Andrew Sackville-West wrote:
>
> > it turns out you can do a reasonable amount of stuff in that busybox
> > shell and if the system is close to booting, you can get it to
> > go. What I was missing, and desperately wanted, was some kind of text
> > editor. I ended up have to do some funky 'cat'ing of files and text
> > input to tweak an encryption-key script. Not fun without an editor,
> > but you sure can do it...
> >
>
> You mean that the busybox doesn't have any editor at all? The busybox
> package (not necessarily the same as the initrd version) has vi and ed.
> No ed in the initrd?
I didn't look real hard, but I did look. I thought of ed, but maybe I
just didn't see it. I can induce a "waiting for root" situation easily
on my laptop... I'll confirm it.
A