[gentoo-user] boot hangs forever at “Loading initial ramdisk...”

[John Blinka]

Hi, Gentooers,

New thread, next obstacle in booting new Asus mobo.

As the subject says, the boot hangs indefinitely.  Output to the screen is

  Booting a command list

Loading Linux 5.10.27-gentoo-x86_64 ...

Loading initial ramdisk ...

And there it stops forever.

The kernel is the latest stable gentoo-sources.  I normally do a custom configuration, but in this instance built it with “genkernel all”, using whatever config genkernel produces. I use grub (grub2), and installed the kernel and initrd with “grub-mkconfig -o /boot/grub/grub.cfg”, as I normally do.

Googling around shows that this problem tends to occur when grub can’t find the initrd.

So  I looked at the grub boot script by pressing “e” just before the boot starts to make sure that grub is looking in the right place for the kernel and for the initrd.  I think it is, since deliberately misspelling either file name with the grub editor causes error messages saying grub can’t find what I told it to look for.  And those error messages do not occur with the boot script that grub generated.

Normally, loading the initrd takes only a few seconds.

How does one debug this situation?

John

[Jack]

I'd start by removing any "quiet" or "splash" from the kernel command
line.    You should be able to see them when you hit "e". I'm not sure
if it will actually help, but it should be a start.

[John Blinka]

On Thu, May 13, 2021 at 7:23 PM Jack <ostr...@users.sourceforge.net> wrote:

I'd start by removing any "quiet" or "splash" from the kernel command
line.    You should be able to see them when you hit "e". I'm not sure
if it will actually help, but it should be a start.

Thanks, but neither one appears.  My command line is

linux  /vmlinuz… root=UUID=… ro loglevel=4 nomodeset

Here I’ve replaced the full name of the kernel and the uuid of the boot partition with ellipses because it’s too tedious to type.  I’ve scrutinized the actual ones for typos and am convinced there are none.  Leaving out the loglevel command doesn’t change the behavior at all.

John

[Dale]

I hate these init thingys and will admit I know little about the
things.  I had a thought tho, could it be that the file system needed to
read the init thingy isn't included somehow or in the kernel maybe?  If
it is pointing to the right place, sounds like it is to me, then it has
to be a read problem I'd think. 

I haven't ever had to use the edit menu on grub2 that I remember.  It
might be worth mentioning that it may have tab completion.  That would
certainly remove a typo if it can complete the kernel or init thingys
file name on its own.  Just a thought.

Going back under my desk now. 

Dale

:-)  :-) 

[Jack]

Given  you say the UUID is for the boot partition, then both the linux and initrd should just have the name of the kernel and initrd files (without leading "/boot",) which sounds like what  you've got.  I'd next wonder if something is missing from the kernel/initrd combination, such as a kernel module necessary for some early part of the boot process or a file system (per Dale's suggestion.)  Assuming that you ran genkernel after booting a live image and chrooting into the new system, then we know the hardware can boot a good kernel/image combo.  Mainly I'm  just thinking out loud here, trying to coax someone's little gray cells into action.

[John Blinka]

On Thu, May 13, 2021 at 9:10 PM Dale <rdale...@gmail.com> wrote:

I hate these init thingys and will admit I know little about the
things.  I had a thought tho, could it be that the file system needed to
read the init thingy isn't included somehow or in the kernel maybe?  If
it is pointing to the right place, sounds like it is to me, then it has
to be a read problem I'd think.

All the uefi stuff is on a fat filesystem.  I would think that something that fundamental (and universally supported) is embedded in the bios.  The grub bootloader itself is on that fat filesystem, and it must have loaded or else I wouldn’t have access to the grub edit facility.  So I think I’m ok on file system support.

I haven't ever had to use the edit menu on grub2 that I remember.  It
might be worth mentioning that it may have tab completion.  That would
certainly remove a typo if it can complete the kernel or init thingys
file name on its own.  Just a thought.

Grub documentation says it does have tab completion.  But the file names, uuids, and other things prone to typos that I referenced are generated by grub, so typos are unlikely to be an issue.  And I’ve checked them meticulously.  They look ok.

Going back under my desk now.

Maybe I’d be less frustrated by this new mobo if I did the same! ;)

John

[John Blinka]

On Thu, May 13, 2021 at 9:12 PM Jack <ostr...@users.sourceforge.net> wrote:

Given  you say the UUID is for the boot partition, then both the linux and initrd should just have the name of the kernel and initrd files (without leading "/boot",) which sounds like what  you've got.  I'd next wonder if something is missing from the kernel/initrd combination, such as a kernel module necessary for some early part of the boot process or a file system (per Dale's suggestion.)  Assuming that you ran genkernel after booting a live image and chrooting into the new system, then we know the hardware can boot a good kernel/image combo.  Mainly I'm  just thinking out loud here, trying to coax someone's little gray cells into action.

In my early linux days, I thought it would be clever to include kernel support for my root filesystem in a module.  Whose code resided on the root filesystem...  That didn’t work, of course, but at least the kernel started to boot and threw out an error message.  Here, I just get complete silence.  So, I doubt that file system support is an issue.

John

[John Covici]

On Thu, 13 May 2021 21:58:25 -0400,
John Blinka wrote:
>
> [1 <text/plain; UTF-8 (quoted-printable)>]

I would look in the grub.cfg and give us exactly what is in the stanza
you are using, including where it thinks the root file system is,
etc. Also, see if there is any genkernel option to get some debugging
info out of the initrd, I know using dracut you can get breakpoints
during the process and see how its doing.

--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?

John Covici wb2una
cov...@ccs.covici.com

[William Kenworthy]

Try https://wiki.ubuntu.com/DebuggingKernelBoot ... I am not sure
genkernel uses that exact name but I did need to find the initramfs boot
log to diagnose a failure in a genkernel initramfs at one time.

BillK

[Peter Humphrey]

On Friday, 14 May 2021 00:06:07 BST John Blinka wrote:

> How does one debug this situation?

Just coming to this belatedly, not having noticed what may be a connection until now. I have an Asus X99-A motherboard, and I never got grub to work at all. I use a combination of efibootmgr and bootctl to manage my boot process. (Bootctl is in sys-boot/systemd-boot; that's the only systemd package I have on this openrc system.)

You could look at the handbook [1] for how to set these up. I had to develop my own method by picking bits out of it. I use efibootmgr to create entries in the UEFI BIOS, then bootctl to keep them up to date. Oh, and I had to leave a small, otherwise unused partition before the FAT32 boot partition. From parted:

Number  Start      End        Size      File system     Name  Flags
1      1.00MiB    9.00MiB    8.00MiB
2      9.00MiB    1025MiB    1016MiB   fat32                 boot, esp
3      1025MiB    50176MiB   49151MiB  linux-swap(v1)        swap
4      50176MiB   66560MiB   16384MiB  ext4
5      66560MiB   132096MiB  65536MiB  ext4
[...]

This may be a red herring of course; I wouldn't be too surprised if I'm doing things all wrong, and your motherboard may differ from mine in many ways.

1.  https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Bootloader

--

Regards,

Peter.

[John Blinka]

On Fri, May 14, 2021 at 2:36 AM John Covici <cov...@ccs.covici.com> wrote:

I would look in the grub.cfg and give us exactly what is in the stanza
you are using, including where it thinks the root file system is,
etc.  Also, see if there is any genkernel option to get some debugging
info out of the initrd, I know using dracut you can get breakpoints
during the process and see how its doing.

Here’s what I see when pressing “e” just before the system attempts to boot:

setparams ‘Gentoo GNU/Linux’

        load_video

        if [ “x$grub_platform” = xefi ]; then

                set gfxpayload=keep

        fi

        insmod gzio

        insmod part_gpt

        insmod fat

        set root=‘hd0,gpt2’

        if [ x$feature_platform_search_hint = xy ]; then

          search —no-floppy —fs-uuid —set=root —hint-bios=hd0,gpt2 —hint-baremetal=ahci0, got2 5C75-30A0

        else

          search —no-floppy —fs-uuid —set=root 5C75-30A0

        fi

        echo            ‘Loading Linux 5.10.27-gentoo-x86_64 ...’

        linux             /vmlinuz-5.10.27-gentoo-x86_64 root=UUID=0df096ca-4dc8-4325-9296-7b0ddb67f044 ro loglevel=4 nomodeset

        echo            ‘Loading initial ramdisk ...’

        initrd            /early_ucode.cpio /initramfs-5.10.27-gentoo-x86_64.img

I have checked the uuid and filenames - they are correct.  (hd0,gpt2) makes sense.  There’s only 1 disk connected, it uses gpt, and the second partition is a fat boot partition with the above uuid.  The named files exist on that partition.

I don’t see anything in ‘man genkernel’ that looks like a way to get debug info out of an initrd/initramfs.  Looks like there’s a way to turn it off, so perhaps it’s on by default?

John

[John Blinka]

On Fri, May 14, 2021 at 3:12 AM William Kenworthy <bi...@iinet.net.au> wrote

>
Try https://wiki.ubuntu.com/DebuggingKernelBoot ... I am not sure
genkernel uses that exact name but I did need to find the initramfs boot
log to diagnose a failure in a genkernel initramfs at one time.

That’s an intriguing link.  Exploring it now.

John

[John Blinka]

On Fri, May 14, 2021 at 7:50 AM John Blinka <john....@gmail.com> wrote:

On Fri, May 14, 2021 at 3:12 AM William Kenworthy <bi...@iinet.net.au> wrote

>
Try https://wiki.ubuntu.com/DebuggingKernelBoot ... I am not sure
genkernel uses that exact name but I did need to find the initramfs boot
log to diagnose a failure in a genkernel initramfs at one time.

Unfortunately, made no difference and provided no new information.  

https://www.askapache.com/linux/linux-debugging/ might be too old to be relevant any more, but tried almost everything it suggested to obtain debugging output.  Not one byte of debug info appeared.  Going to try the dracut approach suggested elsewhere.

John 

[John Blinka]

n

On Fri, May 14, 2021 at 2:36 AM John Covici <cov...@ccs.covici.com> wrote:

I would look in the grub.cfg and give us exactly what is in the stanza
you are using, including where it thinks the root file system is,
etc.  Also, see if there is any genkernel option to get some debugging
info out of the initrd, I know using dracut you can get breakpoints
during the process and see how its doing.

Tried dracut.  No change.

Added the kernel command line debug options (#3 in “Identifying your problem area” in ‘man dracut’).  No change.

Feeling peevish, I made a file of random junk using dd if=/dev/random of=initrd.img count=4096.  Then supplied that pile of junk as the initrd.  Again, no change.

Then I supplied a nonexistent file name (xxx.img) as the initrd.  This time I got a complaint:

error: file ‘/xxx.img’ not found.

Press any key to continue...

So, it’s getting as far as wanting to read the initrd, and is smart enough to tell whether the specified initrd actually exists on the specified boot partition.  But it can’t actually be doing anything with the initrd, or it would have objected to the random junk I fed it.

From https://en.m.wikipedia.org/wiki/Initial_ramdisk#Implementation, it appears that grub is in charge of loading both linux and the initrd into memory, then handing execution over to linux along with a pointer to the memory location of the initrd.

I’ve observed that that no booting output comes out of linux, nor any complaints from linux about the nonsense contents I fed it from the random initrd I built.  That suggests to me that grub has failed to load linux and/or the initrd into memory, or that it's failed to hand execution control to linux.

Next step:  learned how to run an interactive grub2 command shell. With full debugging turned on, it looks like grub2 can load the kernel image, and it looks like it loads the initrd as well.  At least there are no complaints and the reported initrd size looks correct.

But when I issue the boot command, grub2 issues a handful of mallocs and does a little token parsing, and then just stops...

So it appears that the boot problem arises right around the handoff from grub2 to linux.  Don’t know whether grub2 or linux has failed.  I don’t know how to get either one to tell me more.

John

[mad.scienti...@tutanota.com]

--"Fascism begins the moment a ruling class, fearing the people may use their political democracy to gain economic democracy, begins to destroy political democracy in order to retain its power of exploitation and special privilege." Tommy Douglas




May 14, 2021, 15:15 by john....@gmail.com:

> n
>
>
> On Fri, May 14, 2021 at 2:36 AM John Covici <> cov...@ccs.covici.com> > wrote:
>
>>
>> I would look in the grub.cfg and give us exactly what is in the stanza
>> you are using, including where it thinks the root file system is,
>> etc.  Also, see if there is any genkernel option to get some debugging
>> info out of the initrd, I know using dracut you can get breakpoints
>> during the process and see how its doing.
>>
>
> Tried dracut.  No change.
>
> Added the kernel command line debug options (#3 in “Identifying your problem area” in ‘man dracut’).  No change.
>
> Feeling peevish, I made a file of random junk using dd if=/dev/random of=initrd.img count=4096.  Then supplied that pile of junk as the initrd.  Again, no change.
>
> Then I supplied a nonexistent file name (xxx.img) as the initrd.  This time I got a complaint:
>
> error: file ‘/xxx.img’ not found.
>
> Press any key to continue...
>
> So, it’s getting as far as wanting to read the initrd, and is smart enough to tell whether the specified initrd actually exists on the specified boot partition.  But it can’t actually be doing anything with the initrd, or it would have objected to the random junk I fed it.
>

> From > https://en.m.wikipedia.org/wiki/Initial_ramdisk#Implementation> , it appears that grub is in charge of loading both linux and the initrd into memory, then handing execution over to linux along with a pointer to the memory location of the initrd.

>
> I’ve observed that that no booting output comes out of linux, nor any complaints from linux about the nonsense contents I fed it from the random initrd I built.  That suggests to me that grub has failed to load linux and/or the initrd into memory, or that it's failed to hand execution control to linux.
>
> Next step:  learned how to run an interactive grub2 command shell. With full debugging turned on, it looks like grub2 can load the kernel image, and it looks like it loads the initrd as well.  At least there are no complaints and the reported initrd size looks correct.
>
> But when I issue the boot command, grub2 issues a handful of mallocs and does a little token parsing, and then just stops...
>
> So it appears that the boot problem arises right around the handoff from grub2 to linux.  Don’t know whether grub2 or linux has failed.  I don’t know how to get either one to tell me more.
>
> John
>

Have you recompiled the kernel?  Could be a random, erroneous write to disk or something in the kernel compile didn't go well.  I'd suggest also rebuilding the initrd and reinstalling grub.  I.e. I think there is likely a kernel compile issue since it doesn't ever launch the kernel succesfully either on autopilot or when you run grub interactive.  Might also recompile grub, perhaps there's a change in compiler options that produces an incompatible (at least partially).  I also suggest the rebuild so you can be sure you have the right initrd and matching kernel.

[Dale]

mad.scienti...@tutanota.com wrote:
>
> Have you recompiled the kernel?  Could be a random, erroneous write to disk or something in the kernel compile didn't go well.  I'd suggest also rebuilding the initrd and reinstalling grub.  I.e. I think there is likely a kernel compile issue since it doesn't ever launch the kernel succesfully either on autopilot or when you run grub interactive.  Might also recompile grub, perhaps there's a change in compiler options that produces an incompatible (at least partially).  I also suggest the rebuild so you can be sure you have the right initrd and matching kernel.
>
>

I had another thought.  Just in case it is a bug with grub that only
affects certain hardware, maybe try a different bootloader?  Maybe try
lilo or some other bootloader that works with your hardware.  I seem to
recall you having EFI so I'm sure there is plenty of those to choose from.

Dale

:-)  :-)

[John Blinka]

On Fri, May 14, 2021 at 7:10 PM <mad.scienti...@tutanota.com> wrote:

>
Have you recompiled the kernel?  Could be a random, erroneous write to disk or something in the kernel compile didn't go well.  I'd suggest also rebuilding the initrd

Yes.  Same problems with several kernels and associated initrds, the latter produced by genkernel or dracut or even some gibberish I pretended was an initrd.  From grub debug output, I believe the problem exists right before the kernel tries to use the initrd.  It’s contents are irrelevant at that stage.

and reinstalling grub.

That may or may not be the answer, but it’s such an obvious step. Will definitely give that a try.

I.e. I think there is likely a kernel compile issue since it doesn't ever launch the kernel succesfully either on autopilot or when you run grub interactive.  Might also recompile grub, perhaps there's a change in compiler options that produces an incompatible (at least partially).  I also suggest the rebuild so you can be sure you have the right initrd and matching kernel.

I don’t think it’s a kernel compile issue.  I just now used efibootmgr to create a uefi entry with kernel command line parameters to define the root fs and initrd.  That worked.  That result focuses the blame on grub.

John

[John Blinka]

On Fri, May 14, 2021 at 7:32 PM Dale <rdale...@gmail.com> wrote:

I had another thought.  Just in case it is a bug with grub that only
affects certain hardware, maybe try a different bootloader?  Maybe try
lilo or some other bootloader that works with your hardware.  I seem to
recall you having EFI so I'm sure there is plenty of those to choose from.

Good thought - I just now got a successful boot using efibootmgr.  Peter Humphries also suggested that somewhere in this thread, but also stated that he couldn’t get grub to work on the same brand mobo (Asus).  I’d almost believe that it’s a grub-Asus thing except for the fact that Sysrescue and Ubuntu both boot successfully. And I think they use grub.  

Maybe time to experiment with a different version or fresh installation of grub.  Except that I’m burnt out on booting.  Think I’ll go outside and dig some holes.

John 

[Peter Humphrey]

On Saturday, 15 May 2021 00:54:02 BST John Blinka wrote:

> I don’t think it’s a kernel compile issue.  I just now used efibootmgr to

> create a uefi entry with kernel command line parameters to define the root

> fs and initrd.  That worked.  That result focuses the blame on grub.

I'm glad that worked. Personally, I'm pleased to have ditched grub altogether.

--

Regards,

Peter Humphrey.

[Todd Goodman]

This is likely not your issue with an integrated Intel GPU, but I was building a new system recently with UEFI, ASUS ROG mobo, and nvidia GPU and had this same issue.

Surprisingly, this turned out to require me to set the simple framebuffer support in the kernel config (I also set the UEFI framebuffer support) or else I would get no screen output after the loading initial ramdisk... message.

Just something I ran into for the first time ever recently

Todd

[John Blinka]

On Sat, May 15, 2021 at 8:03 AM Todd Goodman <t...@bonedaddy.net> wrote:

This is likely not your issue with an integrated Intel GPU, but I was building a new system recently with UEFI, ASUS ROG mobo, and nvidia GPU and had this same issue.

Surprisingly, this turned out to require me to set the simple framebuffer support in the kernel config (I also set the UEFI framebuffer support) or else I would get no screen output after the loading initial ramdisk... message.

Just something I ran into for the first time ever recently

Thanks!  This may actually be the crux of all the issues I’ve had.  I’ve now got this mobo booting gentoo from disk (instead of usb) while using grub as the boot loader.  I have not gotten X11 working.  When I make the kernel modifications advised in https://wiki.gentoo.org/wiki/Xorg/Guide, I get the same black screen after loading the initrd as I did the first time I started wrestling with this beast.  Adding “nomodeset” to the boot command gives me back my screen, but it interferes with X11.  So, I’ll take another look at the kernel config with your experience in mind.

Fingers crossed!

John

[William Kenworthy]

Hi,

    genkernel keeps a very detailed log at /run/iniramfs/gksoreport.txt.
(or similar)

When it exits to the cant find root prompt, type "shell" and you can
read/save the report.

BillK

[John Blinka]

So, the root of my booting problem is Linux doesn’t yet fully support my intel core i5-10400 processor’s uhd 630 graphics.

I’ve been able to boot gentoo-sources-5.10.27 successfully using grub on my new Asus Tuf Gaming B560M-Plus Wifi board equipped with an intel core i5-10400 processor.

If you use the nomodeset kernel parameter, the boot messages scrolling by on the screen look like what you’d expect from a normal grub/gentoo boot.

If you don’t use nomodeset,  a few initial lines of boot progress appear from grub, then stop.  It looks like the boot has hung after the initrd, but it hasn’t.  When the kernel takes over the display, something in the modesetting goes wrong, and nothing is displayed from that point on.  But the system boots fine otherwise.

 

You can’t start X11 even after following the instructions in 

https://wiki.gentoo.org/wiki/Xorg/Guide.  The Xorg log indicates that it can’t find the proper driver.

After searching for the errors reported in the Xorg log, I stumbled on the same problem I’m having in

https://unix.stackexchange.com/questions/642535/xorg-detects-no-displays-with-an-intel-uhd-630.  That article in turn refers to activity in 

https://cgit.freedesktop.org/drm-tip, which is the direct rendering manager development site.  What I gather from poking around there is that the intel 915 video driver has just been patched to solve the problem I’m having, but uncertain whether the patch has made it out to the linux distros.  The problem affects all intel processors with onboard uhd 630 graphics.

I’ve tried gentoo-sources 5.10.27 and 5.11.21.  Both have the video problem.  I’ve also tried 5.12.4.  Can’t get it to boot.  Have also tried Ubuntu-20.04.2.0-desktop. It works fine on my older intel boxes, but displays the same video  problem on my new board/cpu.  Haven’t verified it, but guessing none of these have the uhd 630 patch.

So, my previous suspects - secure boot, motherboard, grub, and my customized kernel configs - are all innocent and working fine.  Just have to wait for the patched intel driver to show up in gentoo-sources.

Thanks for all the helpful advice I received!  Couldn’t have gotten this far without it. 

John 

[Neil Bothwick]

On Mon, 17 May 2021 05:08:44 -0400, John Blinka wrote:

> So, my previous suspects - secure boot, motherboard, grub, and my
> customized kernel configs - are all innocent and working fine. Just
> have to wait for the patched intel driver to show up in gentoo-sources.

If you haven't already, and no one else has, post a request for it n
b.g.o.


--
Neil Bothwick

Top Oxymorons Number 39: Almost exactly