Bug#678694: preseed_fetch fails with relative url

[Diederik de Haas]

Package: preseed
Severity: normal
Tags: d-i

Dear Maintainer,
*** Please consider answering these questions, where appropriate ***

* What led up to the situation?
Trying to install Debian Wheezy in a VirtualBox image using preseeding and
trying to use the following line:
d-i preseed/late_command string preseed_fetch \
/scripts/late_command.sh /root/late_command.sh

* What exactly did you do (or not do) that was effective (or
ineffective)?
- Create a VirtualBox (VB) image
- Assign debian-wheezy-DI-a1-amd64-businesscard.iso to the CDROM
drive
- Start the VB image and when presented with d-i start screen,
press TAB to add "auto=true url=my.server.com DEBCONF_DEBUG=5" so it uses my
preseed file from http://my.server.com/d-i/wheezy/./preseed.cfg
I only added DEBCONF_DEBUG=5 when deciding to report a bug,
without it, it also failed.
- At the very end (after GRUB install) I get an error msg saying
'Execution of preseeded command "preseed_fetch
/scripts/late_command.sh /root/late_command.sh" failed with exit
code 2.'
Going to Ctl+Alt+F4, I see the error "can't open
'/usr/lib/fetch-url//./scrips/late_command.sh'"

* What was the outcome of this action?
The requested file was not downloaded to the requested location.

Relevant section of /var/log/installer/syslog:

Jun 23 17:21:03 debconf: --> GET preseed/late_command
Jun 23 17:21:03 debconf: <-- 0 preseed_fetch /scripts/late_command.sh
/root/late_command.sh
Jun 23 17:21:04 preseed: running preseed command
preseed/late_command: preseed_fetch /scripts/late_command.sh
/root/late_command.sh
Jun 23 17:21:04 log-output: /bin/fetch-url:
Jun 23 17:21:04 log-output: .:
Jun 23 17:21:04 log-output: line 33:
Jun 23 17:21:04 log-output: can't open
'/usr/lib/fetch-url//./scripts/late_command.sh'
Jun 23 17:21:04 log-output:
Jun 23 17:21:04 debconf: --> SUBST preseed/command_failed COMMAND
preseed_fetch /scripts/late_command.sh /root/late_command.sh
Jun 23 17:21:04 debconf: Adding [COMMAND] -> [preseed_fetch
/scripts/late_command.sh /root/late_command.sh]
Jun 23 17:21:04 debconf: <-- 0
Jun 23 17:21:04 debconf: --> SUBST preseed/command_failed CODE 2
Jun 23 17:21:04 debconf: Adding [CODE] -> [2]
Jun 23 17:21:04 debconf: <-- 0
Jun 23 17:21:04 debconf: --> INPUT critical preseed/command_failed
Jun 23 17:21:04 debconf: <-- 0 question will be asked

* What outcome did you expect instead?
That the file was downloaded to the specified location (and the next
step would be to execute it).

This bug may have similarities with bug #573138, but when trying to
use preseed/include it failed to retrieve that file with the
following error:
"The file needed for preconfiguration could not be retrieved from
http://my.server.com/d-i/wheezy/./core-pkgs.cfg. The installation
will proceed in non-automated mode."
Ctl+Alt+F4 showed the following line (ie no error msg):
"preseed: successfully loaded preseed file from
http://my.server.com/d-i/wheezy/./preseed.cfg"

When loading the various urls in my browser, they all loaded fine.
I haven't tried with absolute urls, since that would 'destroy' the
portability of my preseed file(s).

My preseed.cfg file:
=====================================
#### Contents of the preconfiguration file (for wheezy)
### Localization

# Preseeding only locale sets language, country and locale.
#d-i debian-installer/locale string en_US

# The values can also be preseeded individually for greater
# flexibility.
d-i debian-installer/language string en
d-i debian-installer/country string NL
d-i debian-installer/locale string en_US.UTF-8
d-i console-keymaps-at/keymap select us
d-i keyboard-configuration/xkb-keymap select us
# Optionally specify additional locales to be generated.
#d-i localechooser/supported-locales en_US.UTF-8, nl_NL.UTF-8

d-i mirror/country string nl
d-i mirror/http/hostname string ftp.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string http://192.168.1.5:3142
#d-i mirror/http/proxy seen false

#Additional repositories
d-i apt-setup/local0/repository string http://ftp.debian.org/debian
sid main
d-i apt-setup/local0/comment string Unstable repository
d-i apt-setup/local0/source boolean false

d-i passwd/root-password password root
d-i passwd/root-password-again password root

d-i passwd/user-fullname string Debian User
d-i passwd/username string debian
d-i passwd/user-password password debian
d-i passwd/user-password-again password debian


d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/expert_recipe string \
boot-root :: \
8000 8000 8000 ext4 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ / } . \
100 589 -1 linux-swap \
method{ swap } format{ } .

d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

# Participate in the package usage survey?
d-i popularity-contest/participate boolean false

# Choose software to install:
tasksel tasksel/first multiselect standard
# Additional pkgs to install:
d-i pkgsel/include string openssh-server vim htop bzip2 tmux zerofree

# Run late_command script to set additional things up
#d-i preseed/include string core-pkgs.cfg
d-i preseed/late_command string preseed_fetch
/scripts/late_command.sh /root/late_command.sh

# Grub
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true

# Finish
d-i finish-install/reboot_in_progress note

=====================================


-- System Information:
Debian Release: wheezy/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash



--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

[Nils Bronstert]

Hello,

I have the same problem. It seems to be still present in the installer
for Debian 9 (Stretch). If you use preseed_fetch it does not use a
relative path to the URL used for the main preseed script. Instead the
absolute path is used from the file system.

Seems that the patch from John Morrissey could fix it.

--
Nils Bronstert

[Jonathan Leroy - Inikup]

This bug is still present in Buster.
Very simple workaround:

d-i preseed/early_command string echo "$(debconf-get preseed/url)" >
/var/run/preseed.last_location

--
Jonathan Leroy

[Johannes Schauer Marin Rodrigues]

Hi Diederik,

thank you for notifying me of this bug report over at #1010878.

On Sat, 19 Oct 2019 15:53:22 +0200 Jonathan Leroy - Inikup <jona...@inikup.com> wrote:
> This bug is still present in Buster.
> Very simple workaround:
>
> d-i preseed/early_command string echo "$(debconf-get preseed/url)" >
> /var/run/preseed.last_location

You probably already saw Phils reply to #1010878 but if anybody else is reading
this I also wanted to quickly record what I did to avoid the problem.

So, I'm running d-i in qemu with preseed/url=http://10.0.2.2:8000 and serve my
current directory with "python3 -m http.server". Then in
d-i/bullseye/preseed.cfg I have:

d-i preseed/run string fetch.sh
d-i preseed/late_command string sh /tmp/setup-testbed /target

My d-i/bullseye/fetch.sh then runs preseed_fetch:

#!/bin/sh
preseed_fetch setup-testbed /tmp/setup-testbed

And that will download d-i/bullseye/setup-testbed into /tmp/setup-testbed to
make it available for my preseed/late_command.

Thanks!

cheers, josch

[Diederik de Haas]

Hi Josch,

On Friday, 13 May 2022 13:02:51 CEST Johannes Schauer Marin Rodrigues wrote:
> You probably already saw Phils reply to #1010878 but if anybody else is
> reading this I also wanted to quickly record what I did to avoid the
> problem.

It appears that Phil send it to you directly (and not to the BTS)?
I did see that parts you quoted of that though, so thanks for that :-)

As it looks like this bug (678694) is likely a documentation issue,
I'm pasting (most of) Philip's reply into this bug as well as it describes
how it is supposed to be used:

Quoting Philip Hands (2022-05-13 11:52:05)
> Johannes Schauer Marin Rodrigues <jo...@debian.org> writes:
> > I'm confused. Shouldn't preseed_fetch try to obtain the setup-testbed
> > relative to the preseed file it just obtained?
>
> preseed_fetch gets things relative to the place the previous
> preseed_fetch got things from, which is one step later than you are
> expecting because it has not really been setup while you're in the
> context of the initial preseed.cfg.
>
> See: https://salsa.debian.org/installer-team/preseed/-/blob/master/README.preseed_fetch
>
> The point is that you need to have done a preseed/include or a
> preseed/run (which can be relative paths) in order to have populated the
> previous download path, whereas I think you are trying to run the
> preseed_fetch while we're still on the first time through the code, in
> the loop above this line:
>
> https://salsa.debian.org/installer-team/preseed/-/blob/master/preseed.sh#L120
>
> I guess there should be some documentation that's easier to find than
> the above README, to say that you should not use relative paths for
> preseed_fetch from the early_command (and possibly the late_command?) in
> preseed.cfg itself, but only from within files that were grabbed via at
> least one preseed/run or preseed/include -- or something like that.
> ...
> Here's another example that may be instructive:
>
> https://hands.com/d-i/bug/846002/
>
> (which is referred to here: https://bugs.debian.org/846002#304 )

So it appears that I didn't fully/properly understand how the preseed_fetch
functionality works and/or how it is supposed to be used and therefor did it
incorrectly, causing it to fail for me.

Cheers,
Diederik