Debian live installer problems

[Arjun Krishnan]

Hi

I've been trying to get a debian installer working on my usb stick. The stick has an EFI partition 

and grub installed. The debian live cd iso is stored under a folder called `/boot/iso`. The grub.cfg contains entries of the form

    set imgdevpath='/dev/disk/by-uuid/<uuid>'

    menuentry '[loopback]debian stretch 9.1 livecd' {

         set isofile='/boot/iso/debian-live-9.1.0-amd64-cinnamon+nonfree.iso'

         loopback loop $isofile

         linux (loop)/live/vmlinuz-4.9.0-3-amd64 boot=live components findiso=${imgdevpath}/${isofile}

         initrd (loop)/live/initrd.img-4.9.0-3-amd64

    }

       menuentry "Debian livecd 9.1 graphical installer" {

        set isofile='/boot/iso/debian-live-9.1.0-amd64-cinnamon+nonfree.iso'

        loopback loop $isofile

        linux  (loop)/d-i/gtk/vmlinuz append video=vesa:ywrap,mtrr vga=788 "findiso=${imgdevpath}/${isofile}"

        initrd (loop)/d-i/gtk/initrd.gz

       }

Once I get to the boot screen and try to run the graphical installer, it fails after loading the kernel. But the live cd does boot. However, the live cd that I booted above (cinnamon+nonfree) does not have a way to run the debian installer after it has booted.

I tried apt install debian-installer, but it gives the following error.

    $ sudo apt install debian-installer-launcher

    $ sudo debian-installer-launcher

    no suitable d-i initrd image found, aborting

    umount /lib/live/installer: mountpoint not found

I dont know where this bug should be filed, since I cannot file it against a package. So I thought i'd ask here. I'm happy to file a bug and leave it at that. In any case I debootstrap when I usually install things.

Arjun

[Jimmy Johnson]

On 08/19/2017 08:49 PM, Arjun Krishnan wrote:
> Hi
>
> I've been trying to get a debian installer working on my usb stick. The
> stick has an EFI partition

<snip>

> I dont know where this bug should be filed, since I cannot file it against
> a package. So I thought i'd ask here. I'm happy to file a bug and leave it
> at that. In any case I debootstrap when I usually install things.
>
> Arjun

Probably here: linux.debian.devel.cd

As for your install I would recommend you using the non-free net-install
iso, with it you will get the option to install a desktop and servers too.

The DVD installs have many bugs filed against them and maybe fixed with
the next release.

cheers,
--
Jimmy Johnson

Ubuntu 14.04 LTS - KDE 4.13.3 - Intel G3220 - EXT4 at sda1
Registered Linux User #380263

[Mario Castelán Castro]

On 2017-08-19 21:49 -0600 Arjun Krishnan <arj...@gmail.com> wrote:
>Once I get to the boot screen and try to run the graphical installer, it
>fails after loading the kernel. But the live cd does boot. However, the
>live cd that I booted above (cinnamon+nonfree) does not have a way to run
>the debian installer after it has booted.

The installer needs to find its own ISO image. The non-live installer will
only search by default in the root directories of your file-systems, but
not in the subdirectories. Maybe this is the case with the live installer
as well. Try putting the ISO image in the root directory (“/”).

I always do a new install rather than an in-place upgrade, to get rid of
the garbage, especially packages that I install and configuration files
that I write which I forget about and no longer need.

I have installed the latest 3 Debian releases (or maybe more) using the
“hd-image”
<https://www.debian.org/releases/stretch/amd64/ch05s01.html.en#boot-initrd>
vmlinuz and initrd.gz whose purpose is to look for the ISO image in an
existing file-system and load it. Note that these are non-live installers.

The only obstacle I have found is that the procedure to verify the vmlinuz
and initrd.gz is *not* documented, so I will describe it below. You
*should* verify your initrd.gz and vmlinuz if you follow this procedure. It
is done in several steps. Change the URIs to the mirror of your choice. I
assume that you download all files to the same working directory.

Verify that the hash of

(1): Download <http://ftp.mx.debian.org/debian/dists/stretch/Release>
and <http://ftp.mx.debian.org/debian/dists/stretch/Release.gpg>. Install
the package “debian-archive-keyring” in your current system. Verify the
signature with “gpg --no-default-keyring
--keyring /usr/share/keyrings/debian-archive-keyring.gpg --verify
Release.gpg”.

(2): Download
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/SHA256SUMS>.
Verify with “grep "^ $(sha256sum SHA256SUMS | cut -b
1-66).*main/installer-amd64/20170615+deb9u1/images/SHA256SUMS$" Release”.
The verification is successful if it displays a line of text from the file
“Release” and the exit status is 0.

(3): Download
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/hd-media/gtk/initrd.gz>
and
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/hd-media/gtk/vmlinuz>.
Verify with “sed -nE
'/hd-media\/gtk\/(initrd.gz|vmlinuz)$/{s/^([[:xdigit:]]*).*\/([^/]*)$/\1
\2/;p}' SHA256SUMS | sha256sum --strict -c”. The verification is
successful if sha256sum exits with 0 status and prints output reporting
that the hash matches for these 2 files. This is for the graphical
installer. Remove the “/gtk” part in the URL and the sed script if you
want the text installer.

Also, I DISCOURAGE USING NON-FREE SOFTWARE BECAUSE YOU GIVE UP MUCH OF
YOUR COMPUTING AUTONOMY AND ENCOURAGE THE UNETHICAL PRACTICE OF WRITING
PROPRIETARY SOFTWARE.

Regards.

[Arjun Krishnan]

On Sun, Aug 20, 2017 at 9:15 AM, Mario Castelán Castro <mario...@yandex.com> wrote:

On 2017-08-19 21:49 -0600 Arjun Krishnan <arj...@gmail.com> wrote:
>Once I get to the boot screen and try to run the graphical installer, it
>fails after loading the kernel. But the live cd does boot. However, the
>live cd that I booted above (cinnamon+nonfree) does not have a way to run
>the debian installer after it has booted.

The installer needs to find its own ISO image. The non-live installer will
only search by default in the root directories of your file-systems, but
not in the subdirectories. Maybe this is the case with the live installer
as well. Try putting the ISO image in the root directory (“/”).

It doesn't seem to be finding the iso even after moving to the root directory in the usb drive.

Which iso did you use? I used the netinst iso, and the hd-media kernel and initrd, both of which didnt

work.

[Mario Castelán Castro]

On 2017-08-20 09:59 -0600 Arjun Krishnan <arj...@gmail.com> wrote:
>> The installer needs to find its own ISO image. The non-live installer
>> will only search by default in the root directories of your
>> file-systems, but not in the subdirectories. Maybe this is the case
>> with the live installer as well. Try putting the ISO image in the root
>> directory (“/”).
>It doesn't seem to be finding the iso even after moving to the root
>directory in the usb drive.
>Which iso did you use? I used the netinst iso, and the hd-media kernel and
>initrd, both of which didnt
>work.

I used the ISO of the first non-live DVD. Make sure you use an ISO of the
same release than the initrd.gz and vmlinuz.


If you want to do a net install, you do not need any ISO. As the Debian
installation manual (which you should have at least glanced over) says:

“If you intend to use the hard drive only for booting and then download
everything over the network, you should download the
netboot/debian-installer/amd64/initrd.gz file and its corresponding kernel
netboot/debian-installer/amd64/linux. This will allow you to repartition
the hard disk from which you boot the installer, although you should do so
with care.”

To download and verify the initrd.gz and vmlinuz, use my already given
instructions (I have *not* verified this variation), except that in step
(1) substitute the URIs with
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/netboot/gtk/debian-installer/amd64/linux>
and
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/netboot/gtk/debian-installer/amd64/initrd.gz>
and in step (3), substitute the command given with:

“sed -nE
'/netboot\/gtk\/debian-installer\/amd64\/(initrd.gz|linux)$/{s/^([[:xdigit:]]*).*\/([^/]*)$/\1
\2/;p}' SHA256SUMS | sha256 --strict -c”.

-----
Another way, IIRC, is that you can instead extract the initrd.gz and
vmlinuz from the netinstall ISO (these also do not need the ISO image,
not even the netboot one, again IIRC). I think I did this with Debian 8,
but I do not remember the details.

[Arjun Krishnan]

On Sun, Aug 20, 2017 at 12:38 PM, Mario Castelán Castro <mario...@yandex.com> wrote:

On 2017-08-20 09:59 -0600 Arjun Krishnan <arj...@gmail.com> wrote:
>> The installer needs to find its own ISO image. The non-live installer
>> will only search by default in the root directories of your
>> file-systems, but not in the subdirectories. Maybe this is the case
>> with the live installer as well. Try putting the ISO image in the root
>> directory (“/”).
>It doesn't seem to be finding the iso even after moving to the root
>directory in the usb drive.
>Which iso did you use? I used the netinst iso, and the hd-media kernel and
>initrd, both of which didnt
>work.

I did use an iso of the same release, but I guess it's not so straightforward for me for the following reason:

I have an efi partition on my usb drive, which loads grub. This setup allows me to just add isos to the usb drive

and have grub chainload the kernel and initrd from the iso. This works on mint, ubuntu, arch, etc.

Now grub.cfg has entries that look like this, where debian-squeeze.iso is on the root directory of the usb drive.

menuentry 'debian netinst' {

   set iso='debian-squeeze.iso'

   loopback loop $iso

   linux (loop)/install.amd/vmlinuz findiso=$iso

   initrd (loop)/install.amd/initrd.gz

}

So thinking I had the wrong initrd like you suggested, I copied the initrd and vmlinuz to the root partition of the usb, and changed these to

menuentry 'debian netinst' {

   set iso='debian-squeeze.iso'

   linux /vmlinuz-hd-image findiso=$iso

   initrd /initrd-hd-image.gz

}

The iso happens to be in the root partition of the drive, so are the initrd and vmlinuz. They're just being loaded weirdly, I suppose.

I used the ISO of the first non-live DVD. Make sure you use an ISO of the
same release than the initrd.gz and vmlinuz.

I did ensure this.

If you want to do a net install, you do not need any ISO. As the Debian
installation manual (which you should have at least glanced over) says:

I did read over it, but it doesn't seem to cover my specific situation, or at least I

ve not been able to figure it out. 

[Mario Castelán Castro]

On 2017-08-20 13:36 -0400 Arjun Krishnan <arj...@gmail.com> wrote:
>Now grub.cfg has entries that look like this, where debian-squeeze.iso is
>on the root directory of the usb drive.

Also (again I forgot in the previous message): I was assuming that you
were trying to install Debian 9 “stretch”, not Debian 6 “Squeeze”.

In any case, I recommend to keep the file name of the ISO image as-is, in
case the initrd search for this specific file name.

[Mario Castelán Castro]

On 2017-08-20 13:36 -0400 Arjun Krishnan <arj...@gmail.com> wrote:

>So thinking I had the wrong initrd like you suggested, I copied the initrd
>and vmlinuz to the root partition of the usb

*Which* “initrd”? There are many of them. The ones *inside* the ISO image
does not work for loading the ISO image from an existing partition.

If you want to load the ISO image from an existing partition, you must use
the hd-media ones, which I have described already.

It is not clear to me what the current problem is. In your previous
message you mentioned what you did, but you did not mention whether it
worked or not. If it did not work, then provide details.

>> If you want to do a net install, you do not need any ISO. As the Debian
>> installation manual (which you should have at least glanced over)
>> says:
>
>I did read over it, but it doesn't seem to cover my specific situation, or

>at least Ive not been able to figure it out.

The quote in my previous message is from
“https://www.debian.org/releases/stable/amd64/ch04s04.html.en”. Doesn't
this cover your use case?

--
Do not eat animals, respect them as you respect people.
https://duckduckgo.com/?q=how+to+(become+OR+eat)+vegan

[Pascal Hambourg]

Le 20/08/2017 à 19:53, Mario Castelán Castro a écrit :
> On 2017-08-20 13:36 -0400 Arjun Krishnan <arj...@gmail.com> wrote:
>> So thinking I had the wrong initrd like you suggested, I copied the initrd
>> and vmlinuz to the root partition of the usb
>
> *Which* “initrd”? There are many of them. The ones *inside* the ISO image
> does not work for loading the ISO image from an existing partition.
>
> If you want to load the ISO image from an existing partition, you must use
> the hd-media ones, which I have described already.

Also, IIRC, the ISO file must be on a FAT filesystem, because at that
stage the installer can only mount FAT or ISO9660.

> It is not clear to me what the current problem is.

Neither to me. From the first post I understood that the boot process
failed after loading the kernel, i.e. before loading the initrd and
running the kernel, so whether the initrd could find the ISO file was
irrelevant.

[Arjun Krishnan]

On Sun, Aug 20, 2017 at 2:13 PM, Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:

Le 20/08/2017 à 19:53, Mario Castelán Castro a écrit :

On 2017-08-20 13:36 -0400 Arjun Krishnan <arj...@gmail.com> wrote:

So thinking I had the wrong initrd like you suggested, I copied the initrd
and vmlinuz to the root partition of the usb

*Which* “initrd”? There are many of them. The ones *inside* the ISO image
does not work for loading the ISO image from an existing partition.

If you want to load the ISO image from an existing partition, you must use
the hd-media ones, which I have described already.

Also, IIRC, the ISO file must be on a FAT filesystem, because at that stage the installer can only mount FAT or ISO9660.

 

Oh! This does make a difference, because all my linux isos were on an ext4 filesystem. But so are the kernel and the initrd.

Do you know why the debian installer fails to support this, but the debian live cd and other ubuntu installers all manage to boot off the iso? To elaborate, why does loading the iso as a loop device, and then loading the kernel and initrd off of that work for the debian live cd, but not for the debian installer?

It is not clear to me what the current problem is.

 

Let me try to clarify the problem.

1. I have a usb stick that has an efi partition and a normal ext4 partition. The efi partition contains the grub efi shim. The normal ext4 partition contains grub, grub.cfg and several 

    linux isos. 

2. I've tried loading linux and initrd using grub: mount the iso as a loopdevice, then boot off the kernel and initrd in on the iso. The livecd works when loaded this way, but the netinst cd does not. 

3. As Mario suggested, I downloaded the *stretch* hd-image initrd and linux. I put them all (linux, initrd, iso) in the root partition of the ext4 partition, and had grub load the kernel and the initrd.

It loads the kernel (I think) and then it hangs. 

 

Neither to me. From the first post I understood that the boot process failed after loading the kernel, i.e. before loading the initrd and running the kernel, so whether the initrd could find the ISO file was irrelevant.

I don't know where in the process it fails. It says "failed to find video mode" and hangs.

The use case in the debian installation manual does appear to cover my use case. But perhaps it doesn't work because it can only load a FAT filesystem?

[Mario Castelán Castro]

On 2017-08-20 14:58 -0600 Arjun Krishnan <arj...@gmail.com> wrote:
>On Sun, Aug 20, 2017 at 2:13 PM, Pascal Hambourg <pas...@plouf.fr.eu.org>
>wrote:

>> Also, IIRC, the ISO file must be on a FAT filesystem, because at that
>> stage the installer can only mount FAT or ISO9660.
>
>Oh! This does make a difference, because all my linux isos were on an ext4
>filesystem. But so are the kernel and the initrd.
>

>[...]

>
>The use case in the debian installation manual

><https://www.debian.org/releases/stable/amd64/ch04s04.html.en> does appear

>to cover my use case. But perhaps it doesn't work because it can only load
>a FAT filesystem?

This is incorrect. The initrd.gz and vmlinuz of “hd-image” *CAN* load from
an ext4 filesystem. That is is how I installed the system I am using right
now, as I write this message! However, I installed from the original
Stretch release (not the 9.1 release). Of course, there is a small chance
that a bug has been introduced in the latest release that prevents loading
from an ext4 file-system, but I judge the probability as very small.

>Do you know why the debian installer fails to support this, but the debian
>live cd and other ubuntu installers all manage to boot off the iso? To
>elaborate, why does loading the iso as a loop device, and then loading the
>kernel and initrd off of that work for the debian live cd, but not for the
>debian installer?

My guess is that you are not following the instructions in the Debian
installation guide complemented by my commentary. You seem to have your own
idea of how to do things, which to me appears to be your own wild guess.

For example, you say that you load the ISO with the loop option in GRUB.
Why are you doing this? GRUB does NOT need the ISO, only the vmlinuz and
initrd.gz from “hd-media”, which are (AFAIK) not found anywhere within the
ISO. The ISO contains a different initrd.gz and vmlinuz.

Let us recall the steps to install Debian 9 from a USB drive or hard
disk:

(1): Downloaded and verify “debian-9.1.0-amd64-DVD-1.iso” (from here:
<https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-9.1.0-amd64-DVD-1.iso>
or another mirror) and placed it WITHOUT renaming (I do not know if
renaming is allowed, but let us assume that it is not) in the root
directory of your ext4 partition in the USB drive.

(2): Download and verify
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/hd-media/gtk/vmlinuz>
and
<http://ftp.mx.debian.org/debian/dists/stretch/main/installer-amd64/20170615+deb9u1/images/hd-media/gtk/initrd.gz>.
I described how to verify them in a previous message so I am not going to
repeat it.

(3): Copy these files to the directory within the USB drive where the GRUB
configuration files are. Probably this
is “$PATH_TO_ROOT_OF_DRIVE/boot/grub”. Set up your “grub.cfg” to load
*THESE* initrd and vmlinuz (*NOT* the ones from the ISO image).

(4): Reboot and install.

If this procedure fails, please describe the error message, or whatever
is at the screen at the moment of failure.

[Arjun Krishnan]

My guess is that you are not following the instructions in the Debian
installation guide complemented by my commentary. You seem to have your own
idea of how to do things, which to me appears to be your own wild guess.

This is not the case. There's no need to be so negative and disrespectful, Mario, you're not Linus Torvalds. 

We're all only trying to learn here. However, I appreciate the time you took to try and help.

I finally got the debian installer to work, and here is how. I wanted an easy-to-maintain usb stick that would install all of the different linux distributions (see https://wiki.archlinux.org/index.php/Multiboot_USB_drive). The problem was that my usb stick was running off 

an efi grub. It had two partitions, one efi containing the grub shim, and one ext4. I have a bunch of different linux isos with different distributions distributions loaded in the ext4 partition, and one single grub.cfg file that loads the installers from these isos. The debian installer was the only one giving me trouble. It turned out that the debian installer needed an extra grub needed a module called `efi_gop` to be loaded. I don't know what it does, and would appreciate any information.

The following grub2 entry boots the debian net installer. In this case, I'd copied the hd-image initrd and vmlinuz on to the ext4 partition as 

stated in the debian install guide.

menuentry 'debian stretch 9.1 netinst' {

    insmod efi_gop

    search --no-floppy --set=root --fs-uuid e84f9c3d-3021-45b0-b2c6-da7852c3c151 --hint hd0,gpt2

    linux /vmlinuz vga=788 --- quiet

    initrd /initrd.gz

}

Once the installer loaded, it searched for isos on the usb stick and loaded the packages it needed from there.

In fact, it is not necessary to get the initrd and kernel from the hd-image folder *at all*, and the following menuentry works just as well.

menuentry '[loopback]debian stretch 9.1 xfce-cd config' {

        insmod efi_gop

        # this is the path to the ext4 partition on the usb drive

        set imgdevpath="/dev/disk/by-uuid/e84f9c3d-3021-45b0-b2c6-da7852c3c151"

        set isofile='/boot/iso/debian-9.1.0-amd64-xfce-CD-1.iso'

        loopback loop $isofile 

        linux (loop)/install.amd/gtk/vmlinuz desktop=xfce fromiso=${imgdevpath}/${isofile}

        initrd (loop)/install.amd/gtk/initrd.gz

}

Notice here that grub2 is loading the ramdisk and linux kernel that reside on the iso. If I removed the `insmod efi_gop` statement from either of these, the installer would fail to boot.

What I don't understand is why the efi_gop module is needed for the Debian installer, but not for the Debian live cd.