Bug#1004348: hurd: FTBFS on hurd-i386 locally

[Svante Signell]

Source: hurd
Version: 0.9.git20211230-5
Severity: important
Tags: patch
User: debia...@lists.debian.org
Usertags: hurd

Hi,

Currently hurd FTBFS on GNU/Hurd due to a problem with the
Makeconf/mkinstalldirs scripts.

From the build log:
make -C libshouldbeinlibc install
make[3]: Entering directory '/home/.../hurd-0.9.git20211230/build-deb/
libshouldbeinlibc'
mkdir /home
mkdir: cannot create directory ‘/home’: File exists
mkdir /home/.../hurd-0.9.git20211230/debian/tmp/lib
mkdir /home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu
../../Makeconf:307: recipe for target '/home/.../hurd-
0.9.git20211230/debian/tmp/lib/i386-gnu' failed
make[3]: *** [/home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu]
Error 1
make[3]: Leaving directory '/home/.../hurd-0.9.git20211230/build-
deb/libshouldbeinlibc'
../Makefile:280: recipe for target 'libshouldbeinlibc-install' failed
make[2]: *** [libshouldbeinlibc-install] Error 2

On the buildd the build succeeds:
make[3]: Entering directory '/<<PKGBUILDDIR>>/build-
deb/libshouldbeinlibc'
mkdir /<<PKGBUILDDIR>>/debian/tmp/lib
mkdir /<<PKGBUILDDIR>>/debian/tmp/lib/i386-gnu
mkdir /<<PKGBUILDDIR>>/debian/tmp/usr
mkdir /<<PKGBUILDDIR>>/debian/tmp/usr/include
where 
I: NOTICE: Log filtering will replace 'build/hurd-52h0Hx/hurd-
0.9.git20211230' with '<<PKGBUILDDIR>>'

Maybe building hurd in a chroot would also succeed. This has not been
attempted yet.

The attached patch, Makeconf.patch, has been used to successfully build
hurd on GNU/Hurd locally. Alternately one could modify mkinstalldirs
accordingly since MKINSTALLDIRS = $(top_srcdir)/mkinstalldirs, and
replacing mkdir with mkdir -p, as given in mkinstalldirs.patch.

With this patch hurd builds fine too. It might be the preferred
solution.

From the build log:
make -C libshouldbeinlibc install
make[3]: Entering directory '/home/hurd-0.9.git20211230/build-
deb/libshouldbeinlibc'
mkdir -p /home/.../hurd-0.9.git20211230/debian/tmp/lib
mkdir -p /home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu
mkdir -p /home/.../hurd-0.9.git20211230/debian/tmp/usr
mkdir -p /home/.../hurd-0.9.git20211230/debian/tmp/usr/include

Thanks!

[Samuel Thibault]

Hello,

Svante Signell, le mar. 25 janv. 2022 16:15:46 +0100, a ecrit:

> make[3]: Entering directory '/home/.../hurd-0.9.git20211230/build-deb/
> libshouldbeinlibc'
> mkdir /home
> mkdir: cannot create directory ‘/home’: File exists
> mkdir /home/.../hurd-0.9.git20211230/debian/tmp/lib
> mkdir /home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu

Is this really unmodified source code? I don't see how there could be an

mkdir /home

triggered by the build rules you are patching:

> --- a/Makeconf 2022-01-25 10:57:58.000000000 +0100
> +++ b/Makeconf 2022-01-25 10:57:41.000000000 +0100
> @@ -240,7 +240,7 @@
> all: $(install-targets)
> install: $(DESTDIR)$(installationdir) $(addprefix $(DESTDIR)$(installationdir)/,$(install-targets))
> $(DESTDIR)$(installationdir):
> - @$(MKINSTALLDIRS) $@
> + -@$(MKINSTALLDIRS) $@
> $(addprefix $(DESTDIR)$(installationdir)/,$(installable)): $(DESTDIR)$(installationdir)/%: %
> $(INSTALL_PROGRAM) $(INSTALL-$<-ops) $< $@
> else
> @@ -255,7 +255,7 @@
> install-headers: $(DESTDIR)$(includedir)/$(installhdrsubdir) $(addprefix $(DESTDIR)$(includedir)/$(installhdrsubdir)/,$(installhdrs))
>
> $(DESTDIR)$(includedir)/$(installhdrsubdir): $(DESTDIR)$(includedir)
> - @$(MKINSTALLDIRS) $@
> + -@$(MKINSTALLDIRS) $@
>
> # Arrange to have the headers installed locally anytime we build the library.
> # Not quite perfect, but at least it does end up getting done; and once done
> @@ -304,7 +304,7 @@
>
> # Making installation directories
> $(addprefix $(DESTDIR),$(installationdirlist)): %:
> - @$(MKINSTALLDIRS) $@
> + -@$(MKINSTALLDIRS) $@
>
> # Building the target
> ifneq ($(makemode),misc)

None of these are to be /home

> --- a/mkinstalldirs 2021-12-30 15:16:15.000000000 +0100
> +++ b/mkinstalldirs 2022-01-25 15:41:19.000000000 +0100
> @@ -22,9 +22,9 @@
> esac
>
> if test ! -d "$pathcomp"; then
> - echo "mkdir $pathcomp" 1>&2
> + echo "mkdir -p $pathcomp" 1>&2
>
> - mkdir "$pathcomp" || lasterr=$?
> + mkdir -p "$pathcomp" || lasterr=$?
>
> if test ! -d "$pathcomp"; then
> errstatus=$lasterr

The -d test just above makes it so that since /home already exists, no
mkdir would be called.

> make[3]: Entering directory '/<<PKGBUILDDIR>>/build-
> deb/libshouldbeinlibc'
> mkdir /<<PKGBUILDDIR>>/debian/tmp/lib
> mkdir /<<PKGBUILDDIR>>/debian/tmp/lib/i386-gnu
> mkdir /<<PKGBUILDDIR>>/debian/tmp/usr
> mkdir /<<PKGBUILDDIR>>/debian/tmp/usr/include

Here there is not even an "mkdir" printed, so it's not the patch that
fixed it, but something else, which is what you should rather be after.

I'd say revert your patch, and check e.g. with stat why exactly the

"test ! -d $pathcomp"

wouldn't say that /home is already a directory.

And check with e.g. echo what rule exactly produces such

mkdir /home

on your system, there is really no reason for it to be there.

Samuel

[Svante Signell]

On Tue, 2022-01-25 at 16:38 +0100, Samuel Thibault wrote:
> Hello,
>
> Svante Signell, le mar. 25 janv. 2022 16:15:46 +0100, a ecrit:
> > make[3]: Entering directory '/home/.../hurd-0.9.git20211230/build-
> > deb/
> > libshouldbeinlibc'
> > mkdir /home
> > mkdir: cannot create directory ‘/home’: File exists
> > mkdir /home/.../hurd-0.9.git20211230/debian/tmp/lib
> > mkdir /home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu
>
> Is this really unmodified source code? I don't see how there could be
> an
>
> mkdir /home
>
> triggered by the build rules you are patching:

Yes, unmodified sources. Found on two up-to-date images already. Could
this be related to the /proc/5/stat bug?

[Samuel Thibault]

Svante Signell, le mar. 25 janv. 2022 17:04:41 +0100, a ecrit:

I don't see how that could be.

Samuel

[Svante Signell]

On Tue, 2022-01-25 at 17:06 +0100, Samuel Thibault wrote:
> Svante Signell, le mar. 25 janv. 2022 17:04:41 +0100, a ecrit:
> > On Tue, 2022-01-25 at 16:38 +0100, Samuel Thibault wrote:
> > > Svante Signell, le mar. 25 janv. 2022 16:15:46 +0100, a ecrit:
> > > >

> > > > mkdir /home
> > > > mkdir: cannot create directory ‘/home’: File exists

> > >

> > > Is this really unmodified source code? I don't see how there
> > > could be
> > > an
> > >
> > > mkdir /home
> > >
> > > triggered by the build rules you are patching:
> >
> > Yes, unmodified sources. Found on two up-to-date images already.
> > Could
> > this be related to the /proc/5/stat bug?
>
> I don't see how that could be.

Maybe not.

BTW: When you (or somebody else) have confirmed/not confirmed building
the latest hurd successfully on an up-to-date image (no chroot, no
buildd) we can continue this discussion. If not this discussion ends
here.

Thanks!

[Samuel Thibault]

Svante Signell, le mar. 25 janv. 2022 17:52:13 +0100, a ecrit:

> BTW: When you (or somebody else) have confirmed/not confirmed building
> the latest hurd successfully on an up-to-date image (no chroot, no
> buildd) we can continue this discussion.

When I hack on the hurd package I don't use a chroot or a buildd.

I can confirm again that building the package with a mere

dpkg-buildpackage

works fine for me. This is as non-root user, I tried both on the /
partition, and on a separate partition. I also tried upgrading the
20210812 image to the latest packages, it built successfully.
I do get the /proc/5/stat error, so that doesn't seem to be the trigger.

So yes, it seems that the investigation has to happen on your systems.

Samuel

[Svante Signell]

On Tue, 2022-01-25 at 19:43 +0100, Samuel Thibault wrote:
>
> I can confirm again that building the package with a mere
>
>  dpkg-buildpackage
>
> works fine for me. This is as non-root user, I tried both on the /
> partition, and on a separate partition. I also tried upgrading the
> 20210812 image to the latest packages, it built successfully.
> I do get the /proc/5/stat error, so that doesn't seem to be the
> trigger.
>
> So yes, it seems that the investigation has to happen on your
> systems.

Thanks!
But on a third image I get:

mkdir: cannot create directory ‘/home’: File exists

Something is fishy...

[Samuel Thibault]

Svante Signell, le mar. 25 janv. 2022 22:15:50 +0100, a ecrit:

As I already wrote, first make sure exactly *what* is producing this
call. That way you'll see where the fish comes from.

Samuel

[Svante Signell]

On Tue, 2022-01-25 at 22:16 +0100, Samuel Thibault wrote:
> >
> > Thanks!
> > But on a third image I get:
> > mkdir: cannot create directory ‘/home’: File exists
> >
> > Something is fishy...
>
> As I already wrote, first make sure exactly *what* is producing this
> call. That way you'll see where the fish comes from.

I'd really appreciate to know the difference of an old image upgraded
to current and a new one??

Thanks!

[Samuel Thibault]

Svante Signell, le mar. 25 janv. 2022 23:40:17 +0100, a ecrit:

No idea. As I've already written, I'm just using the
debian-hurd-20210812.img image, upgraded to current archive.

Samuel

[Svante Signell]

I did add set -x to mkinstalldirs:

make[3]: Entering directory '/home/.../hurd-0.9.git20211230/build-

deb/libshouldbeinlibc'
+ errstatus=0
+ echo :/home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu
+ sed -ne s/^:\//#/;s/^://;s/\// /g;s/^#/\//;p
+ set fnord /home srs DEBs hurd hurd-0.9.git20211230 debian tmp lib
i386-gnu
+ shift
+ pathcomp=
+ pathcomp=/home
+ test ! -d /home
+ echo mkdir /home
mkdir /home
+ mkdir /home

mkdir: cannot create directory ‘/home’: File exists

+ lasterr=1
+ test ! -d /home
+ errstatus=1
+ pathcomp=/home/
...

[Samuel Thibault]

Svante Signell, le mer. 26 janv. 2022 10:31:23 +0100, a ecrit:

> On Tue, 2022-01-25 at 23:42 +0100, Samuel Thibault wrote:
> > Svante Signell, le mar. 25 janv. 2022 23:40:17 +0100, a ecrit:
> > > On Tue, 2022-01-25 at 22:16 +0100, Samuel Thibault wrote:
> > > > >
> > > > > Thanks!
> > > > > But on a third image I get:
> > > > > mkdir: cannot create directory ‘/home’: File exists
> > > > >
> > > > > Something is fishy...
> > > >
> > > > As I already wrote, first make sure exactly *what* is producing
> > > > this
> > > > call. That way you'll see where the fish comes from.
> > >
> > > I'd really appreciate to know the difference of an old image upgraded
> > > to current and a new one??
> >
> > No idea. As I've already written, I'm just using the
> > debian-hurd-20210812.img image, upgraded to current archive.
>
> I did add set -x to mkinstalldirs:
>
> make[3]: Entering directory '/home/.../hurd-0.9.git20211230/build-
> deb/libshouldbeinlibc'
> + errstatus=0
> + echo :/home/.../hurd-0.9.git20211230/debian/tmp/lib/i386-gnu
> + sed -ne s/^:\//#/;s/^://;s/\// /g;s/^#/\//;p
> + set fnord /home srs DEBs hurd hurd-0.9.git20211230 debian tmp lib i386-gnu

Ah ok, that's what /home comes from, mkinstalldirs already splits the
path into separate components.

> + shift
> + pathcomp=
> + pathcomp=/home
> + test ! -d /home

> + echo mkdir /home
> mkdir /home
> + mkdir /home
> mkdir: cannot create directory ‘/home’: File exists

So here the problem is that

test ! -d /home

says that /home is not a directory. Is there anything special about your
/home path? Perhaps show the output of

stat /home

Samuel

[Svante Signell]

On Wed, 2022-01-26 at 17:54 +0100, Samuel Thibault wrote:
>
> So here the problem is that
>
> test ! -d /home
>
> says that /home is not a directory. Is there anything special about your
> /home path? Perhaps show the output of
>
> stat /home

File: /home
Size: 4096 Blocks: 8 IO Block: 8192 directory
Device: 1a0h/416d Inode: 2 Links: 4
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2022-01-27 07:58:34.000000000 +0100
Modify: 2017-01-20 13:14:35.000000000 +0100
Change: 2017-01-20 13:14:35.000000000 +0100
Birth: -

With similar results for the other two boxes.

cat /etc/fstab:
/dev/hd0s6 /home ext2 defaults 0 2
/dev/hd0s2 /home ext2 defaults 0 2
/dev/hd0s2 /home ext2 defaults 0 2

/bin/sh is linked to dash.
dash 0.5.11+git20210903+057cd650a4ed-3

On all boxes the little script
#! /bin/sh
if test ! -d /home; then
echo "/home is not a directory"
else
echo "/home is a directory"
fi
returns : /home is a directory

and:

file /home
/home: directory

Thanks!

[Samuel Thibault]

Svante Signell, le jeu. 27 janv. 2022 12:08:47 +0100, a ecrit:

One additional thing is that make install is run under fakeroot, so
you'll have to also involve fakeroot in your tests.

Samuel

[Svante Signell]

On Thu, 2022-01-27 at 20:23 +0100, Samuel Thibault wrote:
> Svante Signell, le jeu. 27 janv. 2022 12:08:47 +0100, a ecrit:
> > On Wed, 2022-01-26 at 17:54 +0100, Samuel Thibault wrote:
> > > So here the problem is that
> > >
> > > test ! -d /home
> > >
> > > says that /home is not a directory. Is there anything special about your
> > > /home path? Perhaps show the output of
> > >
> > > stat /home
> >
> > File: /home
> > Size: 4096 Blocks: 8 IO Block: 8192 directory
> > Device: 1a0h/416d Inode: 2 Links: 4
> > Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
> > Access: 2022-01-27 07:58:34.000000000 +0100
> > Modify: 2017-01-20 13:14:35.000000000 +0100
> > Change: 2017-01-20 13:14:35.000000000 +0100
> > Birth: -
> >
> >

> > On all boxes the little script
> > #! /bin/sh
> > if test ! -d /home; then
> > echo "/home is not a directory"
> > else
> > echo "/home is a directory"
> > fi
> > returns : /home is a directory
>
> One additional thing is that make install is run under fakeroot, so
> you'll have to also involve fakeroot in your tests.

The packages was built with dpkg-buildpackage -b 2>&1 | tee ../build.log where
fakeroot is automatically called.

But here is one result:
fakeroot stat /home

File: /home
Size: 4096 Blocks: 8 IO Block: 8192 directory

Device: 1a1h/417d Inode: 2 Links: 4

Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)

Access: 2022-01-26 23:12:36.000000000 +0100
Modify: 2019-07-05 23:29:24.000000000 +0200
Change: 2019-07-05 23:29:24.000000000 +0200
Birth: -

fakeroot ./test_for_directory.sh
/home is a directory

[Samuel Thibault]

Svante Signell, le jeu. 27 janv. 2022 23:17:38 +0100, a ecrit:

I'm out of ideas for now. Perhaps you could add the stat call inside the
mkinstalldirs script, to make sure that yes the stat() call does report
non-directory here. And perhaps also call ps there, to perhaps see the
actual situation it's getting run under. Perhaps fakeroot-tcp vs
fakeroot-hurd?

Samuel