Preseeding from USB

[T o n g]

Hi,

If I want to do automated install on Debian (http://www.hps.com/~tpg/
notebook/autoinstall.php) using preseeding configuration file from USB
key, how should I tell the Debian boot loader to look there?

If normally booted, my USB key would be sdc, should I use

preseed/file=/sdc/path/preseed.cfg

or /mnt/sdc or /media/its_label, or...?

Thanks

--
Tong (remove underscore(s) to reply)
http://xpt.sourceforge.net/techdocs/
http://xpt.sourceforge.net/tools/


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/k20iuk$kiq$1...@ger.gmane.org

[Bob Proulx]

T o n g wrote:
> If I want to do automated install on Debian (http://www.hps.com/~tpg/
> notebook/autoinstall.php)

Those directions are doing a network install and are loading the
preseed file from the network.

> using preseeding configuration file from USB
> key, how should I tell the Debian boot loader to look there?

It isn't completely obvious but as stated here:

http://www.debian.org/releases/stable/i386/apbs02.html.en

If you are using initrd preseeding, you only have to make sure a
file named preseed.cfg is included in the root directory of the
initrd. The installer will automatically check if this file is
present and load it.

So for you booting off of USB the simplest thing would be to put the
preseed file in the initrd of your usb boot media. Then you don't
need to specify anything anywhere and it happens automatically. But
if you decide not to do that then the docs go on to say:

For the other preseeding methods you need to tell the installer what
file to use when you boot it. This is normally done by passing the
kernel a boot parameter, either manually at boot time or by editing
the bootloader configuration file (e.g. syslinux.cfg) and adding the
parameter to the end of the append line(s) for the kernel.

In which case it would be the path to the preseed file at the time
that the install is happening.

> If normally booted, my USB key would be sdc, should I use
>
> preseed/file=/sdc/path/preseed.cfg
>
> or /mnt/sdc or /media/its_label, or...?

You would need to determine what the path to the install media would
be at boot time. I would manually boot the installer and then do
Alt-F2 and then return to get a text console. Then I would inspect
the disks there to see where the installer is mounted. Then use that
path.

Or if you specify a network path at install boot time then just as
shown in the doc you were referencing you can specify an http URL path
to it and load it that way. The only downside is that some variables
such as locale and keyboard are used before networking is activated
and so those questions are not automated by this method.

And so you might find it easier to place the preseed file into the
initrd that is on the usb boot media and then have it happen
automatically. Or at the least have the keyboard and locale variables
set there to avoid those questions and then load the rest of the
preseed file from the network. Being on the network it would be more
easily edited and changed.

So your next question I expect would be: How do I get the preseed file
into the initrd on the usb boot media?

A useful trick to know is that the initrd maybe a list of concatenated
gzip'd files. So you don't have to read-modify-write the existing
initrd.gz file. You only need to *append* an additional file to it.

I haven't tried this with wheezy but with squeeze:

cp $PATH_TO_INSTALL_MEDIA/initrd.gz .
mkdir initrd-preseed
cd initrd-preseed
cat >preseed.cfg <<EOF
d-i debian-installer/locale string en_US.UTF-8
d-i console-keymaps-at/keymap select us
d-i netcfg/choose_interface select eth0
EOF
find . -print | cpio -R 0:0 -o -H newc | gzip >> ../initrd.gz

Then replace the usb install media copy of initrd.gz with your new
copy that has the extra preseed file appended to the end of it. Then
when you boot from that usb media the initrd.gz file will be loaded
and it will have the preseed.cfg file in place in the initrd and it
will be automatically loaded.

You will need to understand the paths and make adjustments in the
above but I think that should give you the right idea.

I only load those three variables in my initrd preseed.cfg file. All
other configuration I make in a network available preseed file and I
specify it on the boot line. I can edit that file easily and the
changes are available without needing to rebuild the initrd each and
every time. But once you have the initrd preseed it is certainly
possible to specify everything there.

Make sure you keep a pristine copy of your original initrd.gz file.
That way you can rebuild by appending to the end of the pristine copy
again. I use a read-only iso image for my master copy. Then I make a
copy of it and append to the end of it for my active boot media.

Bob

[T o n g]

Wow, that's a fast and comprehensive answer. I didn't expect the answer
comes back so quickly. Thanks.

On Sun, 02 Sep 2012 15:56:03 -0600, Bob Proulx wrote:

>> If normally booted, my USB key would be sdc, should I use
>>
>> preseed/file=/sdc/path/preseed.cfg
>>
>> or /mnt/sdc or /media/its_label, or...?
>
> You would need to determine what the path to the install media would be
> at boot time. I would manually boot the installer and then do Alt-F2
> and then return to get a text console. Then I would inspect the disks
> there to see where the installer is mounted. Then use that path.

wow, I was planning to boot from CD and put preseed.cfg on USB, which
will definitely make things even more complicated. You mean using Alt-F2
to get a text console even before boot up process begins? IIRC, the text
console will not be available only until the boot up process is almost
half way through. I guess you are right, preseed.cfg has to be on the
boot up media. But still, its path is for the installer, which can be
entirely different than the normal /mnt/... thing.

Now back to the very beginning, is this automated preseeded installation
a feature only provided by kernel? Is the preseeded installer available
only at boot time, or there is a tool that I can use after the system has
booted, like debootstrap. Is it doing installation from the CD or from
the Internet? I mean, do I have to user a certain CD, eg Debian
installation CD, or it has nothing with the CD media so I can put the
proper boot up instruction in my general-purpose, multi-boot USB key? Eg,

kernel linux
append auto=true priority=critical preseed/locale=en_US kbd-chooser/
method=us preseed/url=http://192.168.13.184/preseed-files/preseed.cfg
ramdisk_size=14984 root=/dev/rd/0 initrd=initrd.gz

(copied from http://serverfault.com/questions/143296/how-to-get-http-
preseed-to-work-correctly-on-ubuntu-10-04-lts-lucid)

Furthermore info about my preseed testing: to speed up reboot & testing,
I'm testing with VirtualBox, using ISO file as its CD. The ISO that I'm
testing now is Ubuntu (because that's the one people most blogged about).
That one (Ubuntu 12.04) has problem with DNS resolving at boot up time,
so I was forced to consider providing preseed.cfg from USB, which now I
think maybe even more troublesome. It'd be much better if the boot
process can be decoupled from the preseeded installation. That will make
troubleshooting much much easier.

Thanks

--
Tong (remove underscore(s) to reply)
http://xpt.sourceforge.net/techdocs/
http://xpt.sourceforge.net/tools/


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: http://lists.debian.org/k20o32$kiq$2...@ger.gmane.org

[Bob Proulx]

T o n g wrote:

> Bob Proulx wrote:
> >> If normally booted, my USB key would be sdc, should I use
> >>
> >> preseed/file=/sdc/path/preseed.cfg
> >>
> >> or /mnt/sdc or /media/its_label, or...?
> >
> > You would need to determine what the path to the install media would be
> > at boot time. I would manually boot the installer and then do Alt-F2
> > and then return to get a text console. Then I would inspect the disks
> > there to see where the installer is mounted. Then use that path.
>
> wow, I was planning to boot from CD and put preseed.cfg on USB, which
> will definitely make things even more complicated.

I am sure what you were planning is possible because during
installation time it is possible to load firmware blobs from usb
media. But I don't know how to do it. If you figure it out please
educate me. :-)

> You mean using Alt-F2 to get a text console even before boot up
> process begins? IIRC, the text console will not be available only
> until the boot up process is almost half way through.

It is available after the debian-installer has booted the kernel. It
will then start asking you questions. You can Alt-F2 and get a
console and poke around.

> I guess you are right, preseed.cfg has to be on the boot up media.

It is much easier that way. And the only way to set keyboard and
locale and default network device.

> But still, its path is for the installer, which can be entirely
> different than the normal /mnt/... thing.

Correct.

> Now back to the very beginning, is this automated preseeded installation
> a feature only provided by kernel?

Not by the kernel. By the debian-installer software. The d-i runs as
a program application after the installation environment kernel has booted.

> Is the preseeded installer available only at boot time, or there is
> a tool that I can use after the system has booted, like
> debootstrap.

Questions phrased as, "Is it possible?", can only ever be answered in
the affirmative. It is always possible.

> Is it doing installation from the CD or from the Internet? I mean,
> do I have to user a certain CD, eg Debian installation CD, or it has
> nothing with the CD media so I can put the proper boot up
> instruction in my general-purpose, multi-boot USB key?

It depends upon the installation media. There are CD/DVD images that
install from hard media. There are netinst images that boot from
media and then install packages from the network. The netinst images
are the most useful and are highly recommended. But you can also use
small CD images that are completely self-contained to install a small
system image.

> Eg,
>
> kernel linux
> append auto=true priority=critical preseed/locale=en_US kbd-chooser/
> method=us preseed/url=http://192.168.13.184/preseed-files/preseed.cfg
> ramdisk_size=14984 root=/dev/rd/0 initrd=initrd.gz
>
> (copied from http://serverfault.com/questions/143296/how-to-get-http-
> preseed-to-work-correctly-on-ubuntu-10-04-lts-lucid)

I use:

append vga=788 initrd=debian-installer/i386/initrd.gz console-keymaps-at/keymap=us locale=en_US interface=eth0 hostname=localhost domain=localdomain auto url=http://localwwwserver/debian/preseed.cfg

Usually I set the hostname and domain to public values there. It
depends upon what I am targeting. Using localhost.localdomain is one
internally consistent possibility. Good for a standalone test victim
machine for random software testing.

> Furthermore info about my preseed testing: to speed up reboot & testing,
> I'm testing with VirtualBox, using ISO file as its CD. The ISO that I'm
> testing now is Ubuntu (because that's the one people most blogged about).

And here we are on the debian-user mailing list talking about Ubuntu
again. (me hangs head and shakes head sorrowfully)

> That one (Ubuntu 12.04) has problem with DNS resolving at boot up time,

I am using Debian and the debian-installer uses dhcp to configure the
network. DHCP provides all of the information needed (nameservers)
for host names to be used. I use names in my URL for the preseed
file. It works fine in Debian. :-)

> so I was forced to consider providing preseed.cfg from USB, which now I

Why were you forced? It looks like you used IP addresses okay. IP
addresses would not need a name lookup.

> think maybe even more troublesome. It'd be much better if the boot
> process can be decoupled from the preseeded installation. That will make
> troubleshooting much much easier.

If you are doing an install boot such that you can control the kernel
command line parameters then I think you don't need an initrd preseed
file but can specify those three early variables keyboard, locale,
network interface, on the command line. Then use a preseed file on
your local web server for everything else. That should work just
fine.

I am not using VirtualBox but I am using KVM. It is similar but
different. Here is a command line that I use to do a network
installation onto a virtual machine. (I use 'env -i' because some
environment variable in my environment that I never closed on caused
virt-install to fail. YMMV.) I set the variables earlier in the
script in the obvious way.

env -i HOME=$HOME PATH=$PATH \
virt-install \
--name "$name" \
--ram "$ram" \
--vcpus "$cpus" \
--hvm \
--nographics \
--disk path=/var/lib/libvirt/images/"$name".img,size="$size" \
--os-type linux \
--os-variant debiansqueeze \
--network bridge:br0 \
--location http://localwww/ftp.us.debian.org/debian/dists/squeeze/main/installer-amd64/ \
--extra-args "console=ttyS,9600n8 initrd=debian-installer/i386/initrd.gz console-keymaps-at/keymap=us locale=en_US interface=eth0 hostname=$name domain=$domain auto url=http://localwww/debian/preseed-all.cfg"

Here I am using the --location option. But this works just as well to
boot a netinst cdrom iso image too. And it also works to boot a PXE
boot image too. (But on my network PXE boot installations require
interaction. So I use the above for fully automated installation.)

Then as long as I am here talking... I use preseeding to hook in a
final finishing up installation script.

d-i preseed/late_command string wget -O/target/var/tmp/bootstrap http://localwww/bootstrap/bootstrap; in-target sh /var/tmp/bootstrap

That runs a script and allows me to do additional configuration in
that script. In that script I install additional packages such as
openssh-server and set up host keys and other things. Since I am
creating clones I configure a cloned host key. I rename the host
system there to whatever I am doing at the time. I use the following
to differentiate virtual machines from real machines:

if grep -q "^model name.* QEMU Virtual CPU" /proc/cpuinfo ; then
# This is a virtual machine.

Then as long as I am here talking... If you create enough unique
systems in a short timespan you might run your dhcp server out of
dynamically available addresses. The d-i will need a different IP
address than the finished host system. (This is by design.)
Therefore I configure the dhcp server with quite short lease times for
volatile systems such as d-i. I don't post everything here but as a
hint this is part of my configuration:

class "volatile" {
match if substring (option vendor-class-identifier, 0, 3) = "d-i" or
substring(option vendor-class-identifier, 0, 9) = "PXEClient" or
substring(option vendor-class-identifier, 0, 9) = "Etherboot" or
substring(option vendor-class-identifier, 0, 8) = "anaconda" or
substring(option vendor-class-identifier, 0, 8) = "volatile";
}

And then volatile devices get a short lease time so that the pool of
addresses isn't depleted.

Bob

[T o n g]

OMG, thanks again for the fast and comprehensive answer.

On Sun, 02 Sep 2012 19:33:56 -0600, Bob Proulx wrote:

>> Is it doing installation from the CD or from the Internet? I mean, do I
>> have to user a certain CD, eg Debian installation CD, or it has nothing
>> with the CD media so I can put the proper boot up instruction in my
>> general-purpose, multi-boot USB key?
>
> It depends upon the installation media. There are CD/DVD images that
> install from hard media. There are netinst images that boot from media
> and then install packages from the network. The netinst images are the
> most useful and are highly recommended. But you can also use small CD
> images that are completely self-contained to install a small system
> image.

This automated preseeded installation is a strange beast to me looking on
paper and from other people's blogs, but having tried it, I just realized
it is the same old Debian netinst, just the answers are pre-filled in the
preseed file. Thus,

to put it into my general-purpose, multi-boot USB key, I just need to
extract the vmlinuz and initrd.gz from the netinst ISO, and put a proper
menu entry in my grub boot menu.

> I use preseeding to hook in a final finishing up installation script.
>
> d-i preseed/late_command string wget -O/target/var/tmp/bootstrap
http://localwww/bootstrap/bootstrap; in-target sh /var/tmp/bootstrap
>
> That runs a script and allows me to do additional configuration in
> that script. In that script I install additional packages such as
> openssh-server and set up host keys and other things.

I install openssh-server with d-i pkgsel/include:

# Individual additional packages to install
d-i pkgsel/include string localepurge openssh-server build-essential
rsync less

but I guess to set up host keys and other things, better do them all
together in the late_command hook.

BTW, I highly recommend to install localepurge in pkgsel/include. It use
to be the most annoying package for me during installation, because I
have to answer so many questions. Now I have it all taken care of:

#######################
# Packages Preseeding
#######################
# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
# debconf-get-selections --installer > file
# debconf-get-selections >> file

### Pkg:localepurge

localepurge localepurge/nopurge multiselect en_US
# Really remove all locales?
localepurge localepurge/none_selected boolean false
# Also delete localized man pages?
localepurge localepurge/mandelete boolean true
# localepurge will not take any action
localepurge localepurge/remove_no note
# Inform about new locales?
localepurge localepurge/dontbothernew boolean false
# Display freed disk space?
localepurge localepurge/showfreedspace boolean true
# Display verbose output?
localepurge localepurge/verbose boolean false
# Accurate disk space calculation?
localepurge localepurge/quickndirtycalc boolean true

Thanks again for your kind help,
Really appreciate it.

--
Tong (remove underscore(s) to reply)
http://xpt.sourceforge.net/techdocs/
http://xpt.sourceforge.net/tools/


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: http://lists.debian.org/k214uo$u7e$1...@ger.gmane.org

[T o n g]

On Sun, 02 Sep 2012 19:33:56 -0600, Bob Proulx wrote:

> I am not using VirtualBox but I am using KVM. It is similar but
> different. Here is a command line that I use to do a network

> installation onto a virtual machine. . .

Personally I prefer KVM than VirtualBox, but choosing VirtualBox because
of my friends and for running on Windows. Your script is extremely
helpful, to me at least, even though I have researched/studied KVM
extensively.

Thanks a lot!

--
Tong (remove underscore(s) to reply)
http://xpt.sourceforge.net/techdocs/
http://xpt.sourceforge.net/tools/


--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: http://lists.debian.org/k215qm$u7e$2...@ger.gmane.org

[Brian]

On Sun 02 Sep 2012 at 21:26:44 +0000, T o n g wrote:

> If I want to do automated install on Debian (http://www.hps.com/~tpg/
> notebook/autoinstall.php) using preseeding configuration file from USB
> key, how should I tell the Debian boot loader to look there?
>
> If normally booted, my USB key would be sdc, should I use
>
> preseed/file=/sdc/path/preseed.cfg
>
> or /mnt/sdc or /media/its_label, or...?

1. Make a FAT16 partition on the key and format it with mkfs.vfat. Put
preseed.cfg there.

2. Before booting the installer in expert mode tell it where the preseed
file will be with the kernel parameter

file=/mnt/preseed.cfg

3. Boot. Switch to console 2 with ALT-F2 and list the disks known to the
kernel:

ls -l /dev/sd*

4. Insert the USB stick and determine its device name, /dev/sdX, using
the previous command.

5. Mount the partition you made in step 1.:

mount -t vfat /dev/sdX1 /mnt

6. Switch to the installer screen with ALT-F1 and proceed with the
installation.


You're booting from an ISO on a CD? Why use old technology? Depending on
the ISO you have you could put it on the USB stick and boot from it. A
second partition on the stick could hold any files you need (firmware,
preseed etc) during the intallation.

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: http://lists.debian.org/20120903090321.GG24280@desktop

[Tom H]

If DNS is a problem, why don't you point to the pressed file with an ip address?

--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Archive: http://lists.debian.org/CAOdo=SwRFwu2gWKQv_FWW=9LXdfF3UfZ4vVu...@mail.gmail.com