[gentoo-user] NVMe drive and grub

[pe...@prh.myzen.co.uk]

Hello list,

I have a new box with an NVMe SSD drive attached to the PCI bus via an M.2 interface. The drive shows up as /dev/nvme0n1, with partitions /dev/nvme0n1p1, /dev/nvme0n1p2, ...

After following the instructions in the handbook for a UEFI system, I get as far as a kernel panic. Grub finds the kernel and starts it, but it seems to be passing a null root device name.

Can anyone offer some advice to help me get past this?

A few pints of Wainwright go to the winner. Conditions apply :-)

Rgds
Peter

[Jeremi Piotrowski]

On Thu, Apr 7, 2016 at 5:24 PM, <pe...@prh.myzen.co.uk> wrote:
> Hello list,
>
> I have a new box with an NVMe SSD drive attached to the PCI bus via an M.2 interface. The drive shows up as /dev/nvme0n1, with partitions /dev/nvme0n1p1, /dev/nvme0n1p2, ...
>
> After following the instructions in the handbook for a UEFI system, I get as far as a kernel panic. Grub finds the kernel and starts it, but it seems to be passing a null root device name.
>

What is in your grub.conf? Have you thought about adding an
initramfs and letting it drop you
to its rescue shell so that you can investigate?

[Remy Blank]

pe...@prh.myzen.co.uk wrote on 2016-04-07 17:24:
> I have a new box with an NVMe SSD drive attached to the PCI bus via an M.2 interface. The drive shows up as /dev/nvme0n1, with partitions /dev/nvme0n1p1, /dev/nvme0n1p2, ...
>
> After following the instructions in the handbook for a UEFI system, I get as far as a kernel panic. Grub finds the kernel and starts it, but it seems to be passing a null root device name.
>
> Can anyone offer some advice to help me get past this?

Make sure you have BLK_DEV_NVME compiled into your kernel (not as a
module), and that you pass the right device name as a root FS to the
kernel. It might also help to use a very recent kernel.

FWIW, I'm writing this from a laptop that boots from two NVMe devices in
RAID 1 configuration, with kernel 4.4.5.

-- Remy

[pe...@prh.myzen.co.uk]

Remy Blank <remy....@pobox.com> wrote :

> pe...@prh.myzen.co.uk
> wrote on 2016-04-07 17:24:
> > I have a new box with an NVMe SSD drive attached to the PCI bus via an M.2
> interface. The drive shows up as /dev/nvme0n1, with partitions /dev/nvme0n1p1,
> /dev/nvme0n1p2, ...
> >
> > After following the instructions in the handbook for a UEFI system, I get as
> far as a kernel panic. Grub finds the kernel and starts it, but it seems to be
> passing a null root device name.
> >
> > Can anyone offer some advice to help me get past this?
>
> Make sure you have BLK_DEV_NVME compiled into your kernel (not as a
> module), and that you pass the right device name as a root FS to the
> kernel. It might also help to use a very recent kernel.

Yes, I have that compiled in, and the kernel is 4.1.15-r1.

> FWIW, I'm writing this from a laptop that boots from two NVMe devices in
> RAID 1 configuration, with kernel 4.4.5.

Now, that's just showing off :-)

--
Rgds
Peter

[Peter Humphrey]

Grub.cfg looks all right to me: at least, it does include a sensible root=
value.

I never see a grub screen - it just starts the current kernel. I did wonder
about an initramfs and I'm trying it now. I've also followed Remy's advice
and used gentoo-sources-4.4.6.

So far I've spent about 30 hours scratching my head, clutching at straws and
going round in circles. I'm getting dizzy. :-)

--
Rgds
Peter

linux counter 5290, 1994/04/23

[Poison BL.]

So, you have Grub setup to give a menu, pause, or at least do
something visible, and it's skipping right past that? That makes me
suspect that Grub's not running at all, and that the kernel's being
loaded by UEFI directly.

What files are in /boot/ on your efi partition (preferably identified
with the file command)?

If you have a kernel in there named bootx64.efi, that's likely the culprit.

--
Poison [BLX]
Joshua M. Murphy

[pe...@prh.myzen.co.uk]

Poison BL. <pois...@gmail.com> wrote :

> On Fri, Apr 8, 2016 at 8:21 AM, Peter Humphrey pe...@prh.myzen.co.uk>
> wrote:
> > On Thursday 07 April 2016 17:56:55 Jeremi Piotrowski wrote:
> >> What is in your grub.conf? Have you thought about adding an
> >> initramfs and letting it drop you to its rescue shell so that you can
> >> investigate?
> >
> > Grub.cfg looks all right to me: at least, it does include a sensible root=
> > value.
> >
> > I never see a grub screen - it just starts the current kernel. I did
> wonder
> > about an initramfs and I'm trying it now. I've also followed Remy's advice
> > and used gentoo-sources-4.4.6.
> >
> > So far I've spent about 30 hours scratching my head, clutching at straws
> and
> > going round in circles. I'm getting dizzy. :-)
> >
> > --
> > Rgds
> > Peter
>
> So, you have Grub setup to give a menu, pause, or at least do
> something visible, and it's skipping right past that? That makes me
> suspect that Grub's not running at all, and that the kernel's being
> loaded by UEFI directly.

Ah! Now why didn't I think of that?

> What files are in /boot/ on your efi partition (preferably identified
> with the file command)?

# file /boot/*
/boot/config-4.4.6-gentoo: Linux make config build file, ASCII text
/boot/efi: directory
/boot/grub: directory
/boot/initramfs-genkernel-x86_64-4.4.6-gentoo: XZ compressed data
/boot/System.map-4.4.6-gentoo: ASCII text
/boot/vmlinuz-4.4.6-gentoo: Linux kernel x86 boot executable bzImage, version 4.4.6-gentoo (root@sysresccd) #7 SMP Fri Apr 8 15:51:24 BST 201, RO-rootFS, swap_dev 0x5, Normal VGA

# file /boot/efi/boot/*
/boot/efi/boot/bootx64.efi: Linux kernel x86 boot executable bzImage, version 4.4.6-gentoo (root@sysresccd) #7 SMP Fri Apr 8 15:51:24 BST 201, RO-rootFS, swap_dev 0x5, Normal VGA

> If you have a kernel in there named bootx64.efi, that's likely the culprit.

That's what it's supposed to look like, isn't it?

(I'm using web-mail after booting sysresccd.)

[Poison BL.]

--> /boot/efi/boot/bootx64.efi <--

This one is what your UEFI's loading at boot. The lack of any other
files in /boot/efi/boot/ makes me suspect Grub's not actually
installed in quite the right spot for the UEFI layer to load it. If
you pointed grub2-install at the right place (the second command line
at [1]), I believe you should have grubx64.efi there as well. If you
want to play with getting it working with the uefi stub in the kernel
(the way it's booting now), you will need to add in the kernel command
line options you need to boot into the kernel itself, which you can do
when you configure it. That's how I have the couple uefi systems I've
built running, but it comes with the downside that you can't have two
available, in parallel, at boot to test the new one without locking
yourself out if it's broken (requiring the same external media boot
you're already doing to fix it this time around).

[1]: https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Bootloader#Configuring_GRUB2

[Remy Blank]

Now now, I was merely providing evidence that what you are trying to do
actually works.

... But yeah, it's fast :)

-- Remy

[Neil Bothwick]

On Fri, 8 Apr 2016 13:26:31 -0400, Poison BL. wrote:

> --> /boot/efi/boot/bootx64.efi <--
>
> This one is what your UEFI's loading at boot. The lack of any other
> files in /boot/efi/boot/ makes me suspect Grub's not actually
> installed in quite the right spot for the UEFI layer to load it. If
> you pointed grub2-install at the right place (the second command line
> at [1]), I believe you should have grubx64.efi there as well.

Did you build GRUB with GRUB_PLATFORMS="efi-64"?

As you're using EFI, you may find life easier if you dispense with GRUB
and use Gummiboot. I find it (or the systemd version) far more friendly.


--
Neil Bothwick

Home is where you hang your @.

[pe...@prh.myzen.co.uk]

Neil Bothwick <ne...@digimed.co.uk> wrote :

> On Fri, 8 Apr 2016 13:26:31 -0400, Poison BL. wrote:
>
> > --> /boot/efi/boot/bootx64.efi

> > This one is what your UEFI's loading at boot. The lack of any other
> > files in /boot/efi/boot/ makes me suspect Grub's not actually
> > installed in quite the right spot for the UEFI layer to load it. If
> > you pointed grub2-install at the right place (the second command line
> > at [1]), I believe you should have grubx64.efi there as well.
>
> Did you build GRUB with GRUB_PLATFORMS="efi-64"?

Yes, as instructed in the wiki.

> As you're using EFI, you may find life easier if you dispense with GRUB
> and use Gummiboot. I find it (or the systemd version) far more friendly.

Maybe I should give it a go - thanks for the idea. I did consider it but then I was deterred by its description as "minimalistic".

[pe...@prh.myzen.co.uk]

Pe...@prh.myzen.co.uk wrote :

I did try it, and it worked like a charm! After five days of fruitless attempts to get grub to work, five minutes with gummiboot was all I needed.

Thank you Neil.

I'll write a Solved reply when I have my system built and can access the whole thread.

[Neil Bothwick]

The usual complaint about GRUB2 is that it is too complex, so minimal is good here. I only have it installed to be able to boot my System Rescue ISO in an emergency, I've not found a way to go that without GRUB.
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Mick]

On Saturday 09 Apr 2016 08:22:16 Neil Bothwick wrote:
> On 9 April 2016 06:43:50 BST, pe...@prh.myzen.co.uk wrote:
> > Neil Bothwick <ne...@digimed.co.uk> wrote :
> > > On Fri, 8 Apr 2016 13:26:31 -0400, Poison BL. wrote:
> > > > --> /boot/efi/boot/bootx64.efi
> > > > This one is what your UEFI's loading at boot. The lack of any
> >
> > other
> >
> > > > files in /boot/efi/boot/ makes me suspect Grub's not actually
> > > > installed in quite the right spot for the UEFI layer to load it.

As long as the correct kernel was saved as /boot/efi/boot/bootx64.efi, or the
MoBo set up to boot from /boot/efi/boot/my_NVMe_kernel_x64.efi, you shouldn't
have a problem booting up using the EFI boot stub.

https://www.kernel.org/doc/Documentation/efi-stub.txt

> > If
> >
> > > > you pointed grub2-install at the right place (the second command
> >
> > line
> >
> > > > at [1]), I believe you should have grubx64.efi there as well.
> > >
> > > Did you build GRUB with GRUB_PLATFORMS="efi-64"?
> >
> > Yes, as instructed in the wiki.
> >
> > > As you're using EFI, you may find life easier if you dispense with
> >
> > GRUB
> >
> > > and use Gummiboot. I find it (or the systemd version) far more
> >
> > friendly.
> >
> > Maybe I should give it a go - thanks for the idea. I did consider it
> > but then I was deterred by its description as "minimalistic".
>
> The usual complaint about GRUB2 is that it is too complex, so minimal is
> good here. I only have it installed to be able to boot my System Rescue
> ISO in an emergency, I've not found a way to go that without GRUB.

I have used GRUB2 to boot ISO images from USB drives and internal drives
without problem, but Gummiboot is less fiddly to set up. With GRUB2 you need
to pay attention in setting the path to the ISO correctly, or GRUB2 will error
out of course.

Unfortunately, you can't boot an ISO in true minimalist fashion just by
pointing to it with the EFI boot stub (the ISO needs to be mounted via a
loopback function).

--
Regards,
Mick

[Neil Bothwick]

On Sat, 09 Apr 2016 08:40:46 +0100, Mick wrote:

> > The usual complaint about GRUB2 is that it is too complex, so minimal
> > is good here. I only have it installed to be able to boot my System
> > Rescue ISO in an emergency, I've not found a way to go that without
> > GRUB.
>
> I have used GRUB2 to boot ISO images from USB drives and internal
> drives without problem,

Yes, I do that allthe time. I also use it to boot ISO images from DVDs :)

> but Gummiboot is less fiddly to set up.

Which is why I prefer it. But it is fine to have both installed and
available from the UEFI boot menu. I do this on my laptop to make sure I
can always boot a rescue ISO if I need to. For my other systems, there's
usually a suitable USB stick nearby.

> Unfortunately, you can't boot an ISO in true minimalist fashion just by
> pointing to it with the EFI boot stub (the ISO needs to be mounted via
> a loopback function).

I know all that. What would be nice would be an EFI boot stub that took
care of that. It occurs to me that the System Rescue Cd kernels may be
UEFI compatible, so it may be a matter of unpacking the ISO into /boot to
be able to access it from gummiboot/bootctl.


--
Neil Bothwick

"It compiled? The first screen came up? Ship it!" -- Bill Gates

[R0b0t1]

It is a matter of unpacking the ISO. I got most of the way through doing this before giving up as the ISO has a lot of assumptions built into it.

I ended up just using GRUB.

[Neil Bothwick]

On Sat, 9 Apr 2016 06:14:16 -0500, R0b0t1 wrote:

> It is a matter of unpacking the ISO. I got most of the way through doing
> this before giving up as the ISO has a lot of assumptions built into it.

Reading your posts is like walking into a room partway through a
conversation. Can you please quote what and whom you are replying to to
give some context. I assume, but cannot be certain, that this was a
reply to my post. So unpacking the ISO sort of worked? It does have a
UEFI enabled kernel?

> I ended up just using GRUB.

That's what I'm doing now. I use bootctl by default but GRUB is installed
and available from the EFI menu if I need to boot from the ISO.


--
Neil Bothwick

Welcome to the world of Windows 95. Stay a while -- stay foooreveeer.

[Neil Bothwick]

On Sat, 9 Apr 2016 06:14:16 -0500, R0b0t1 wrote:

> It is a matter of unpacking the ISO. I got most of the way through doing
> this before giving up as the ISO has a lot of assumptions built into it.

It turns out to be a trivial task:

Loop mount the ISO
Copy the kernel, initramfs and filesystem files to /boot/sysrcd
Add this to /boot/loaders/entries

title System Rescue Cd
version 4.7.2
linux /sysrcd/rescue64
options subdir=sysrcd setkmap=uk rootpass=wasspord
initrd /sysrcd/initram.igz

Reboot!

More details at http://www.system-rescue-cd.org/Sysresccd-manual-en_Easy_install_SystemRescueCd_on_harddisk


--
Neil Bothwick

God: What one human uses to persecute another.

[J. Roeleveld]

Remy (and others),

I have been trying to get Gentoo to boot from an NVME device, but I am getting
stuck.

The kernel-config I use will boot when the root-device is on a spinning sata-
disk and I can see the NVME device.
When using the same config with the root-partition on the NVME, it fails,
complaining it can't find the root-partition.

The NVME driver is loaded into the kernel (not module).

Does anyone have a working config for a 4.4.6 kernel or any other version that
is currently in portage?

Along with the boot-options being used?

Many thanks,

Joost

[J. Roeleveld]

Ignore this.
Got it working, had to enable UEFI first for the NVME device in bios....

Problem caused by silly BIOS devs at ASUS....

--
Joost
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Peter Humphrey]

Glad to hear it. You had me scratching my head there, trying to remember
exactly what I did.

> Problem caused by silly BIOS devs at ASUS....

When confronted with new ways of doing things I naturally assume I don't
understand, before I start name calling. :)

--
Rgds
Peter

[J. Roeleveld]

I was scratching mine as well....

> > Problem caused by silly BIOS devs at ASUS....
>
> When confronted with new ways of doing things I naturally assume I don't
> understand, before I start name calling. :)

The bios is configured to boot in UEFI mode.
But apparently, some of the devices are not, by default....

I fail to see the logic in that.

--
Joost

[Peter Humphrey]

Hmm. That is odd. Not everything in my BIOS seems entirely logical either,
but at least I've learned how to use it.

--
Rgds
Peter