Bug#970460: qemu-user: trashes argv[0] breaking multi-call binaries

[Thorsten Glaser]

Package: qemu-user
Version: 1:5.1+dfsg-4
Severity: important
X-Debbugs-Cc: t...@mirbsd.de, debia...@lists.debian.org

I’m attaching a test program that does the following:

• if argv[1] is "-" it just outputs argv[0] and argv[1]
• otherwise it also execve(2)s argv[1] with its argv[0] set to "meow"

I’ve installed the extra packages:
• gcc-arm-linux-gnueabi
• libc6-dev-armel-cross

tglase@tglase-nb:~ $ gcc -Wall -Wextra -o native tst.c
1|tglase@tglase-nb:~ $ ./native ./native
argv[0]: ./native
argv[1]: ./native
calling...
argv[0]: meow
argv[1]: -
not calling
tglase@tglase-nb:~ $ arm-linux-gnueabi-gcc -o cross tst.c -static
tglase@tglase-nb:~ $ ./cross ./cross
argv[0]: ./cross
argv[1]: ./cross
calling...
argv[0]: ./cross
argv[1]: -
not calling
tglase@tglase-nb:~ $ ./cross ./native
argv[0]: ./cross
argv[1]: ./native
calling...
argv[0]: meow
argv[1]: -
not calling
tglase@tglase-nb:~ $ ./native ./cross
argv[0]: ./native
argv[1]: ./cross
calling...
argv[0]: ./cross
argv[1]: -
not calling

As you can see, if the callee of an exec is a qemu-user target,
argv[0] is trashed.

This, among other things, makes qemu-user-static-based buildds unworkable.

-- System Information:
Debian Release: bullseye/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'oldstable-updates'), (500, 'buildd-unstable'), (500, 'unstable'), (500, 'oldstable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.7.0-2-amd64 (SMP w/2 CPU threads)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/lksh
Init: sysvinit (via /sbin/init)

Versions of packages qemu-user depends on:
ii libc6 2.31-3
ii libcapstone3 4.0.1+really+3.0.5-2
ii libgcc-s1 10.2.0-7
ii libglib2.0-0 2.66.0-1
ii libgnutls30 3.6.15-2
ii libstdc++6 10.2.0-7
ii zlib1g 1:1.2.11.dfsg-2

Versions of packages qemu-user recommends:
ii qemu-user-static [qemu-user-binfmt] 1:5.1+dfsg-4

Versions of packages qemu-user suggests:
ii sudo 1.9.1-2

-- no debconf information

[John Paul Adrian Glaubitz]

> On Sep 16, 2020, at 6:57 PM, Thorsten Glaser <t...@mirbsd.de> wrote:
>
> I’m attaching a test program that does the following:
>
> • if argv[1] is "-" it just outputs argv[0] and argv[1]
> • otherwise it also execve(2)s argv[1] with its argv[0] set to "meow"

That’s been fixed upstream and can be configured with the qemu-binfmt.sh script and the option “preserved=yes”.

Also, this would be an upstream bug, not a Debian bug :).

Adrian

[Michael Tokarev]

16.09.2020 20:15, John Paul Adrian Glaubitz wrote:
>
>> On Sep 16, 2020, at 6:57 PM, Thorsten Glaser <t...@mirbsd.de> wrote:
>>
>> I’m attaching a test program that does the following:
>>
>> • if argv[1] is "-" it just outputs argv[0] and argv[1]
>> • otherwise it also execve(2)s argv[1] with its argv[0] set to "meow"
>
> That’s been fixed upstream and can be configured with the qemu-binfmt.sh script and the option “preserved=yes”.

Where it's been fixed? current git version of scripts/qemu-binfmt-conf.sh
does not have 'preserved' option, and if the P flag is set when registering
binfmt, the kernel will _prepend_ additional argv[0] element which is not
expected by current linux-user/main.c code (so the tst.c example will be
running with 3 args, not 2).

Thanks,

/mjt

[Thorsten Glaser]

John Paul Adrian Glaubitz dixit:

>That’s been fixed upstream and can be configured with the
>qemu-binfmt.sh script and the option “preserved=yes”.

$ locate qemu-binfmt.sh | wc
0 0 0

Also, why didn’t you fix that on the m68k and sh4 qemu buildds then? ;-)

Meow,
//mirabilos
--
> Wish I had pine to hand :-( I'll give lynx a try, thanks.

Michael Schmitz on nntp://news.gmane.org/gmane.linux.debian.ports.68k
a.k.a. {news.gmane.org/nntp}#news.gmane.linux.debian.ports.68k in pine

[Andreas Schwab]

openSUSE carries a patch for this issue:

https://build.opensuse.org/package/view_file/Virtualization/qemu/linux-user-add-binfmt-wrapper-for-argv-0.patch?expand=1

Andreas.

--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."

[Michael Tokarev]

16.09.2020 22:01, Andreas Schwab wrote:
> openSUSE carries a patch for this issue:

We can do much easier than a separate wrapper.
Since we register binfmt entries in the same package where
the qemu-user binaries are, we can patch qemu (a one-liner)
to always expect the P flag. I thin I'll go this route.

Thank you for all the findings!

/mjt

[Thorsten Glaser]

Michael Tokarev dixit:

>Since we register binfmt entries in the same package where
>the qemu-user binaries are, we can patch qemu (a one-liner)
>to always expect the P flag. I thin I'll go this route.

Thanks!

bye,
//mirabilos
--
„Cool, /usr/share/doc/mksh/examples/uhr.gz ist ja ein Grund,
mksh auf jedem System zu installieren.“
-- XTaran auf der OpenRheinRuhr, ganz begeistert
(EN: “[…]uhr.gz is a reason to install mksh on every system.”)

[Michael Tokarev]

16.09.2020 22:55, John Paul Adrian Glaubitz wrote:

> It's called "--persistent yes", not preserve, my bad.

It is entirely different thing, --persistent (or --fix-binary. F flag)
is here for quite some time and helps with chroots and the like.

BTW, it turned out not that simple as I thought, the wrapper as
used by SUSE is still needed - or else it's impossible to distinguish
between direct qemu-foo invocation and the same invocation using
binfmt.

/mjt

[Michael Tokarev]

17.09.2020 10:08, John Paul Adrian Glaubitz wrote:

> On 9/16/20 8:42 PM, Thorsten Glaser wrote:
>> John Paul Adrian Glaubitz dixit:
>>
>>> That’s been fixed upstream and can be configured with the
>>> qemu-binfmt.sh script and the option “preserved=yes”.
>>
>> $ locate qemu-binfmt.sh | wc
>> 0 0 0

It is scripts/qemu-binfmt-conf.sh script inside the qemu sources.
It was a fork of debian's d/update-binfmts.sh, updated for other
use cases.

> Well, to be honest, you should never use the Debian QEMU package. It's almost
> always very outdated and would lack important patches like these. It's easier
> to use local builds from git.

You should not use upstream git of qemu, since it too lacks
important patches like this, - please don't suggest people
to use outdated sources.. :)

(just as a matter of fact, debian usually has new version of
qemu the next day it is released, and I usually keep it up
to date in backports. With debian stable and current qemu 5.
we have a bit of delay since there are a few other things to
backport, but we have 5.0 there).

/mjt

[Michael Tokarev]

17.09.2020 10:56, John Paul Adrian Glaubitz wrote:

> On 9/17/20 9:49 AM, Michael Tokarev wrote:
>> You should not use upstream git of qemu, since it too lacks
>> important patches like this, - please don't suggest people
>> to use outdated sources.. :)
>

> I think I'm one of the heaviest users of QEMU inside Debian and if
> I had stuck with the Debian version of QEMU, the m68k and sh4 ports
> would not be able to keep up due to QEMU issues. Laurent will confirm
> the number of bugs I reported and that got fixed.

You were referring to particular example of a patch which is
NOT merged upstream, - not your other heavy usages. And this
is exactly what my joke about - if lack of this patch makes
source outdated, it should not be used, hence upstream git
is also outdated, nothing more nothing less. I wont deny
other issues being addressed upstream all the time, and
I definitely wont fight against Debian stable policy.

>> (just as a matter of fact, debian usually has new version of
>> qemu the next day it is released, and I usually keep it up
>> to date in backports. With debian stable and current qemu 5.
>> we have a bit of delay since there are a few other things to
>> backport, but we have 5.0 there).
>

> Well, the first thing would be to switch QEMU in Debian to finally
> use systemd-binfmt instead of the old binfmt-support package,
> something that has happened in other distributions long ago.

This is debatable. First of all I didn't know binfmt-support is
"old", just checked its pages and description, - nowhere they
mention it is deprecated or something. And to me, I think
"switching to" should happen *in* that package, ie, when
binfmt-support and systemd cooperates, say, binfmt-support prepares
stuff for systemd and does nothing when booted under systemd.
This will make other packages using binfmt-support to work with
the systemd binfmt registration instantly, and things will continue
to work if systemd is not in use (for those who prefer sysvinit
or other init mechanisms, fwiw).

It's trivial to "switch" to systemd binfmt support on qemu part.
But I don't want to do it without a good reason, and I don't
actually see binfmt-support being "bad" in some way, - what's the
problem with it exactly?

> If you are willing to cooperate though, I'm happy to point you to
> all the patches necessary to address all issues that we observed.

I'm definitely interested in things being fixed in the end, tho
it isn't obvious how much backporting should be done to _testing_
version in Debian.

/mjt

[Thorsten Glaser]

Michael Tokarev dixit:

> 17.09.2020 10:56, John Paul Adrian Glaubitz wrote:
>> On 9/17/20 9:49 AM, Michael Tokarev wrote:

>>>17.09.2020 10:08, John Paul Adrian Glaubitz wrote:
>>>> On 9/16/20 8:42 PM, Thorsten Glaser wrote:
>>>>> John Paul Adrian Glaubitz dixit:
>>>>>
>>>>>> That’s been fixed upstream and can be configured with the
>>>>>> qemu-binfmt.sh script and the option “preserved=yes”.
>>>>>
>>>>> $ locate qemu-binfmt.sh | wc
>>>>> 0 0 0
>>>
>>>It is scripts/qemu-binfmt-conf.sh script inside the qemu sources.
>>>It was a fork of debian's d/update-binfmts.sh, updated for other
>>>use cases.

Ah, okay.

>>>> Well, to be honest, you should never use the Debian QEMU package. It's almost
>>>> always very outdated and would lack important patches like these. It's easier
>>>> to use local builds from git.

Erm, excuse me?!?!?!

Of *course*, when developing for (or on, most of the time)
Debian, I use as many tools in the form packaged in Debian
as possible.

When developing *for* Debian (that is, things that eventually
get uploaded), I think it’s correct to even have the *expectation*
that only packaged things get used throughout the chain.

If your qemu buildds have errors, where else would we fix them?
Also, how else would maintainers reproduce them?

I reported this bug *because* your qemu buildds had errors
building mksh and because I was able to reproduce them.

I *really* _expect_ qemu buildds to use the packaged version
of qemu-user, ideally the one from sid.

>>>You should not use upstream git of qemu, since it too lacks
>>>important patches like this, - please don't suggest people
>>>to use outdated sources.. :)

Thanks for the confirmation from the maintainer.

>> I think I'm one of the heaviest users of QEMU inside Debian and if
>> I had stuck with the Debian version of QEMU, the m68k and sh4 ports
>> would not be able to keep up due to QEMU issues. Laurent will confirm
>> the number of bugs I reported and that got fixed.

That asks for trying to work with the Debian maintainer of the
affected package (qemu in this case) to get the bugs not only
fixed but also available to Debian users. (Yes I know, but see
below.)

>>>(just as a matter of fact, debian usually has new version of
>>>qemu the next day it is released, and I usually keep it up
>>>to date in backports. With debian stable and current qemu 5.
>>>we have a bit of delay since there are a few other things to
>>>backport, but we have 5.0 there).

But a buildd can run unstable anyway, and especially with
qemu-user-static this should be a given.

>> Well, the first thing would be to switch QEMU in Debian to finally
>> use systemd-binfmt instead of the old binfmt-support package,
>> something that has happened in other distributions long ago.

Absolutely not! That will break it on all my systems.

> This will make other packages using binfmt-support to work with
> the systemd binfmt registration instantly, and things will continue
> to work if systemd is not in use (for those who prefer sysvinit
> or other init mechanisms, fwiw).

Thanks.

>>>>> Also, why didnb t you fix that on the m68k and sh4 qemu buildds then? ;-)
>>>>
>>>>I did. But I'm updating the QEMU version on the buildds from time to time
>>>>by rebasing with git master and then I drop all the patches that have been
>>>>applied upstream.

Arrgh, no. Please use the packaged version. That avoids
many issues and ensures trust.

>> If you are willing to cooperate though, I'm happy to point you to
>> all the patches necessary to address all issues that we observed.
>
> I'm definitely interested in things being fixed in the end, tho
> it isn't obvious how much backporting should be done to _testing_
> version in Debian.

Thanks for being open to patching the version in Debian. Not all
maintainers are agreeable like that, unfortunately.

I know the burden of carrying local patches, but backports would
have been already accepted upstream, are scheduled to be removed
with a new upstream release, and are less likely to break things.
Patches that fix things for an architecture, whether guest or host,
should be considered IMHO, even if it’s not a release architecture,
at least during normal development, i.e. not just before a freeze
but all the other time.

>> There is also an important glibc patch necessary to unbreak qemu-user
>> that still hasn't been merged in glibc upstream or in Debian's glibc
>> package [1, 2].

>> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916276
>> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=23960

This is the first time I’m being made aware of that. (I had wondered
about the +qemu releases in unreleased.) This looks like another case
of prodding maintainers (and, perhaps, looking whether it can’t be
fixed in qemu as well).

But that’s what uploads to dpo unreleased are for, after all. And
if done with care and following normal Debian rules (and, incidentally,
being made using _only_ tools from Debian) I consider unreleased part
of Debian on that specific architecture (as it’s a ports-only thing
anyway).

bye,
//mirabilos
--
<ch> you introduced a merge commit │<mika> % g rebase -i HEAD^^
<mika> sorry, no idea and rebasing just fscked │<mika> Segmentation
<ch> should have cloned into a clean repo │ fault (core dumped)
<ch> if I rebase that now, it's really ugh │<mika:#grml> wuahhhhhh

[Finn Thain]

On Thu, 17 Sep 2020, Thorsten Glaser wrote:

>
> >>>> Well, to be honest, you should never use the Debian QEMU package.
> >>>> It's almost always very outdated and would lack important patches
> >>>> like these. It's easier to use local builds from git.
>
> Erm, excuse me?!?!?!
>

It's an over-statement. Let's not over-react. Just read "not presently" in
place of "never".

> ...

> >> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916276
> >> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=23960
>
> This is the first time I'm being made aware of that.

Thanks for the link, Adrian, and thanks for raising this issue upstream.
It's a monumental mess.

[Thorsten Glaser]

Dixi quod…

>Michael Tokarev dixit:
>> 17.09.2020 10:56, John Paul Adrian Glaubitz wrote:
>>> There is also an important glibc patch necessary to unbreak qemu-user
>>> that still hasn't been merged in glibc upstream or in Debian's glibc
>>> package [1, 2].
>
>>> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916276
>>> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=23960
>
>This is the first time I’m being made aware of that. (I had wondered
>about the +qemu releases in unreleased.) This looks like another case
>of prodding maintainers (and, perhaps, looking whether it can’t be
>fixed in qemu as well).

I prodded maintainers, see below, but the latter looks like it’ll be
needed.

---------- Forwarded message ----------
From: Thorsten Glaser <t...@debian.org>
Message-ID: <Pine.BSM.4.64L.20...@herc.mirbsd.org>
To: Aurelien Jarno <aure...@aurel32.net>
Cc: 916...@bugs.debian.org
Date: Sat, 19 Sep 2020 19:51:28 +0000 (UTC)
Subject: Re: Bug#916276: glibc: Please add prelimenary patch to fix regression
on qemu-user

Hi,

>The patch is basically replacing the getdents64 syscall by the getdents
>one. This means that applying this patch would make debian differ with
>regards to other distributions in the syscalls that are used for the
>same binaries. In turns it is likely going to affect binaries that are
>using seccomp and only allow the getdents64 and not the getdents one.

ah, indeed.

>I therefore don't think this is not reasonable to include such a fix on
>our glibc. It would fix the qemu-user case but likely break random
>binaries.

OK, thanks for the explanation.

Let’s hope this can be fixed, somehow, in qemu itself and/or the kernel.

bye,
//mirabilos
--
> Hi, does anyone sell openbsd stickers by themselves and not packaged
> with other products?
No, the only way I've seen them sold is for $40 with a free OpenBSD CD.
-- Haroon Khalid and Steve Shockley in gmane.os.openbsd.misc

[Thorsten Glaser]

Hi Adrian,

could you work around the problem by starting an i386 VM on
your amd64 system, and then running the qemu-user buildds on
that? The return values from the syscalls will natively be
correct 32 bit there…

In the meantime, someone found that this bug also hits without
any qemu involvement:


danny.milo at gmail dot com dixit:

>https://sourceware.org/bugzilla/show_bug.cgi?id=23960

>--- Comment #58 from Danny Milosavljevic <danny.milo at gmail dot com> ---
>The title should be changed to be more general because this does not only break
>qemu-user. In fact, mentioning qemu would make it seem that it can be fixed in
>qemu-user--which it can't.
>
>The same happens on aarch64 if running armhf executables (no qemu anywhere).
[…]


It was also found that this cannot be fixed “only” in qemu.
The GNU libc people are discussing…

bye,
//mirabilos
--
16:47⎜«mika:#grml» .oO(mira ist einfach gut....) 23:22⎜«mikap:#grml»
mirabilos: und dein bootloader ist geil :) 23:29⎜«mikap:#grml» und ich
finds saugeil dass ich ein bsd zum booten mit grml hab, das muss ich dann
gleich mal auf usb-stick installieren -- Michael Prokop über MirOS bsd4grml

[Thorsten Glaser]

Package: qemu-user
Version: 1:5.2+dfsg-3
Followup-For: Bug #970460
X-Debbugs-Cc: t...@mirbsd.de

Hi, what’s the status on this? It is still broken in sid.
Could we please get the fix?

-- System Information:
Debian Release: bullseye/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'oldstable-updates'), (500, 'buildd-unstable'), (500, 'unstable'), (500, 'oldstable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.9.0-4-amd64 (SMP w/2 CPU threads)
Kernel taint flags: TAINT_WARN

Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/lksh
Init: sysvinit (via /sbin/init)

Versions of packages qemu-user depends on:

ii libc6 2.31-9
ii libcapstone4 4.0.2-3
ii libgcc-s1 10.2.1-6
ii libglib2.0-0 2.66.4-1
ii libgnutls30 3.7.0-5
ii libstdc++6 10.2.1-6
ii zlib1g 1:1.2.11.dfsg-2

Versions of packages qemu-user recommends:

ii qemu-user-static [qemu-user-binfmt] 1:5.2+dfsg-3

qemu-user suggests no packages.

-- no debconf information

[Michael Tokarev]

26.01.2021 02:33, Thorsten Glaser wrote:
>
> Hi, what’s the status on this? It is still broken in sid.
> Could we please get the fix?

Mmm. Is there a fix? If there is, I for one don't know it.

Thanks,

/mjt

[Thorsten Glaser]

Hi Michael,

> Mmm. Is there a fix? If there is, I for one don't know it.

AIUI using a wrapper around the qemu binary that adds the
option (I would say which but the qemu-m68k-static manpage
documents qemu 0.9.0 and so doesn’t have it *cough*) to the
invocation, then change the binfmt-misc registration to add
the P flag and call this wrapper script.

Pending a better fix, of course, but that probably involves
changes to the kernel from what I’ve seen.

Thanks,
//mirabilos
--
<dileks> ch: good, you corrected yourself. ppl tend to tweet such news
immediately, sth. like "grml devs seem to be buyable" <ch> dileks: we
_are_. if you throw enough money in our direction, things will happen
<mika> everyone is buyable, it's just a matter of price <mrud> and now
comes [mira] and uses this as a signature ;0 -- they asked for it…

[Helge Deller]

This bug affects the hppa buildd servers as well.

Helge

[Michael Tokarev]

26.01.2021 18:38, Thorsten Glaser пишет:

> Hi Michael,
>
>> Mmm. Is there a fix? If there is, I for one don't know it.
>
> AIUI using a wrapper around the qemu binary that adds the
> option (I would say which but the qemu-m68k-static manpage
> documents qemu 0.9.0 and so doesn’t have it *cough*) to the
> invocation, then change the binfmt-misc registration to add
> the P flag and call this wrapper script.
>
> Pending a better fix, of course, but that probably involves
> changes to the kernel from what I’ve seen.

The problem with the wrapper is that it effectively nullifies
the F flag of binfmt. That is, with F and the binfmt interpreter
being the qemu binary directly, we can use regular, non-static,
qemu-user, or qemu-user-static, and run the thing from outside
of the foreign chroot.

But with the wrapper, we have to have the actual qemu-user binary
runnable within the foreign chroot *too*, - only the wrapper will
be able to reside outside of the chroot, but it will run the
actual qemu binary *within* that chroot.

So in the end it seems this approach with the wrapper breaks more
than it fixes.

I can think of another hack. Qemu-user binary may look at its
name and if it sees some magic prefix (eg, qemu-foo-binfmt-trigger),
it will assume it is run from within the binfmt subsystem with
the P flag in effect. Yes it is hacky, but it *might* work.

Dunno how good it is to perform such experiments this late in
the bullseye development cycle...

What do you all think ?

/mjt

[Michael Tokarev]

13.02.2021 13:19, Michael Tokarev wrote:
...

> I can think of another hack. Qemu-user binary may look at its
> name and if it sees some magic prefix (eg, qemu-foo-binfmt-trigger),
> it will assume it is run from within the binfmt subsystem with
> the P flag in effect. Yes it is hacky, but it *might* work.

Actually this is something I like. Yes it is hackish but it works.
So I implemented this, using /usr/libexec/qemu-binfmt/foo-binfmt-P
symlinks to ../../bin/qemu-foo[-static], and detecting "-binfmt-P"
prefix in qemu-user/main.c.

The only downside of this is that now, for the first time in many
years, binfmt interpreter for qemu has changed. Dunno how well it
is for the users, but the only way to find out is to try it.
Let's see..

/mjt

[Thorsten Glaser]

Michael Tokarev dixit:

> 13.02.2021 13:19, Michael Tokarev wrote:

>>The problem with the wrapper is that it effectively nullifies
>>the F flag of binfmt. That is, with F and the binfmt interpreter
>>being the qemu binary directly, we can use regular, non-static,
>>qemu-user, or qemu-user-static, and run the thing from outside
>>of the foreign chroot.
>>
>>But with the wrapper, we have to have the actual qemu-user binary
>>runnable within the foreign chroot *too*, - only the wrapper will

Oh? I’m always copying the qemu-user-static binaries into the
chroot and you’re really telling me I didn’t need to? õÕ

>> I can think of another hack. Qemu-user binary may look at its
>> name and if it sees some magic prefix (eg, qemu-foo-binfmt-trigger),
>> it will assume it is run from within the binfmt subsystem with
>> the P flag in effect. Yes it is hacky, but it *might* work.
>
> Actually this is something I like. Yes it is hackish but it works.
> So I implemented this, using /usr/libexec/qemu-binfmt/foo-binfmt-P
> symlinks to ../../bin/qemu-foo[-static], and detecting "-binfmt-P"
> prefix in qemu-user/main.c.

Ah, with the P flag the argv[0] of the interpreter is not changed,
so this can work. Yes, this sounds like a really good idea.

Thanks,
//mirabilos
--
Yay for having to rewrite other people's Bash scripts because bash
suddenly stopped supporting the bash extensions they make use of
-- Tonnerre Lombard in #nosec

[Michael Tokarev]

Control: tag -1 pending

14.02.2021 00:56, Thorsten Glaser wrote:
> Michael Tokarev dixit:
>> 13.02.2021 13:19, Michael Tokarev wrote:
>
>>> The problem with the wrapper is that it effectively nullifies
>>> the F flag of binfmt. That is, with F and the binfmt interpreter
>>> being the qemu binary directly, we can use regular, non-static,
>>> qemu-user, or qemu-user-static, and run the thing from outside
>>> of the foreign chroot.
>>>
>>> But with the wrapper, we have to have the actual qemu-user binary
>>> runnable within the foreign chroot *too*, - only the wrapper will
>
> Oh? I’m always copying the qemu-user-static binaries into the
> chroot and you’re really telling me I didn’t need to? õÕ

This is not needed since v.2.12. So this is buster, but not stretch.
This is fix-binary binfmt-misc flag and #868030 . Many users got
used to that already in buster and buster-backports :)

>>> I can think of another hack. Qemu-user binary may look at its
>>> name and if it sees some magic prefix (eg, qemu-foo-binfmt-trigger),
>>> it will assume it is run from within the binfmt subsystem with
>>> the P flag in effect. Yes it is hacky, but it *might* work.
>>
>> Actually this is something I like. Yes it is hackish but it works.
>> So I implemented this, using /usr/libexec/qemu-binfmt/foo-binfmt-P
>> symlinks to ../../bin/qemu-foo[-static], and detecting "-binfmt-P"
>> prefix in qemu-user/main.c.
>
> Ah, with the P flag the argv[0] of the interpreter is not changed,
> so this can work. Yes, this sounds like a really good idea.

Thank you! And it works well too. I plan to walk over the pile of
security issues tomorrow and hopefully upload a new version too.

Thanks,

/mjt

[Helge Deller]

On 2/13/21 11:58 PM, Michael Tokarev wrote:
>>>> I can think of another hack. Qemu-user binary may look at its
>>>> name and if it sees some magic prefix (eg, qemu-foo-binfmt-trigger),
>>>> it will assume it is run from within the binfmt subsystem with
>>>> the P flag in effect. Yes it is hacky, but it *might* work.
>>>
>>> Actually this is something I like. Yes it is hackish but it works.
>>> So I implemented this, using /usr/libexec/qemu-binfmt/foo-binfmt-P
>>> symlinks to ../../bin/qemu-foo[-static], and detecting "-binfmt-P"
>>> prefix in qemu-user/main.c.

It's not fully clear to me yet, how this solves all problems,
so I'm really looking forward to your patches!

For usage with buildd chroots, what will then be written to /proc/sys/fs/binfmt_misc/* ?
Currently I see:
interpreter /usr/bin/qemu-foo-static
flags OCF
With your idea, it seems you will then need to drop the "F" flag (which means
you basically revert fix-binary binfmt-misc flag and #868030) ?
If so, you will then need to have the "foo-binfmt-P" symlink and qemu-foo-static
inside the chroots too (which mostly prevents usage of non-static qemu-foo binaries).

I've picked up Laurents kernel patch:
https://patchwork.kernel.org/project/linux-fsdevel/patch/20200128132539....@vivier.eu/
into my parisc kernel git tree for inclusion into kernel v5.12 (if nobody
objects this happens next week or the week after).
But in case we now have a viable other solution I prefer to drop it again.
Or this your approach too hackish or has other downsides (like loosing F flags),
so that we generally still prefer an "clean" kernel patch as proposed by Laurent?

Helge

[Thorsten Glaser]

Helge Deller dixit:

> For usage with buildd chroots, what will then be written to
> /proc/sys/fs/binfmt_misc/* ?
> Currently I see:
> interpreter /usr/bin/qemu-foo-static
> flags OCF

AIUI:
interpreter /usr/libexec/qemu-binfmt/foo-binfmt-P
flags OCFP

> With your idea, it seems you will then need to drop the "F" flag (which means

No, that’s the beauty of it.

> so that we generally still prefer an "clean" kernel patch as proposed by
> Laurent?

That won’t be available on pre-bookworm systems anyway,
so we’ll carry the -binfmt-P solution for a while.

bye,
//mirabilos
--
<diogenese> Beware of ritual lest you forget the meaning behind it.
<igli> yeah but it means if you really care about something, don't
ritualise it, or you will lose it. don't fetishise it, don't
obsess. or you'll forget why you love it in the first place.

[Thorsten Glaser]

reopen 970460
found 970460 1:5.2+dfsg-5
thanks

Hi again,

unfortunately, this seems to be not fixed, but I can’t figure out why.
When I call the -binfmt-P variant manually, everything works, and it
clearly uses its own argv[0], not /proc/self/exe, to determine what
to do, but when the kernel runs it, it seems to not work:

tglase@tglase-nb:~ $ ./native ./native
argv[0]: ./native
argv[1]: ./native
calling...
argv[0]: meow
argv[1]: -
not calling

tglase@tglase-nb:~ $ ./cross ./cross
argv[0]: ./cross
argv[1]: ./cross
calling...
argv[0]: ./cross
argv[1]: -
not calling
tglase@tglase-nb:~ $ ./cross ./native
argv[0]: ./cross
argv[1]: ./native
calling...
argv[0]: meow
argv[1]: -
not calling
tglase@tglase-nb:~ $ ./native ./cross
argv[0]: ./native
argv[1]: ./cross
calling...
argv[0]: ./cross
argv[1]: -
not calling

tglase@tglase-nb:~ $ file native cross
native: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=62b1ec37932a96a28761c2fc139fe47a89491ec6, for GNU/Linux 3.2.0, not stripped
cross: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, BuildID[sha1]=116c6def65aa69d58a53030c3d21a6a2a7d5107b, for GNU/Linux 3.2.0, not stripped
tglase@tglase-nb:~ $ cat /proc/sys/fs/binfmt_misc/qemu-arm
enabled
interpreter /usr/libexec/qemu-binfmt/arm-binfmt-P
flags: POCF
offset 0
magic 7f454c4601010100000000000000000002002800
mask ffffffffffffff00fffffffffffffffffeffffff
tglase@tglase-nb:~ $ (exec -a meow ./native -)

argv[0]: meow
argv[1]: -
not calling

tglase@tglase-nb:~ $ (exec -a meow ./cross -)

argv[0]: ./cross
argv[1]: -
not calling

tglase@tglase-nb:~ $ /usr/libexec/qemu-binfmt/arm-binfmt-P meow ./cross -

argv[0]: meow
argv[1]: -
not calling

tglase@tglase-nb:~ $ (exec -a moo /usr/libexec/qemu-binfmt/arm-binfmt-P meow ./cross -)
Error while loading meow: No such file or directory
1|tglase@tglase-nb:~ $ realpath /usr/libexec/qemu-binfmt/arm-binfmt-P
/usr/bin/qemu-arm-static
tglase@tglase-nb:~ $ (exec -a arm-binfmt-P /usr/bin/qemu-arm-static meow ./cross -)

argv[0]: meow
argv[1]: -
not calling

tglase@tglase-nb:~ $ strace -o foo -f ./native ./cross

I’ve tried looking at that trace but can’t find anything related…
attaching it, maybe you have better luck?

The interpreter seems to be set up correctly, too.

bye,
//mirabilos
--
«MyISAM tables -will- get corrupted eventually. This is a fact of life. »
“mysql is about as much database as ms access” – “MSSQL at least descends
from a database” “it's a rebranded SyBase” “MySQL however was born from a
flatfile and went downhill from there” – “at least jetDB doesn’t claim to
be a database” (#nosec) ‣‣‣ Please let MySQL and MariaDB finally die!

[Yunqiang Su]

On Tue, 16 Feb 2021 13:11:15 +0000 (UTC) Thorsten Glaser <t...@mirbsd.de> wrote:
> Debian FTP Masters dixit:
>
> > * fix the brown-paper bag bug: wrong argument order
> > in the linux-user-binfmt patch (really closes: #970460)
>
> Confirmed, thanks!

The new kernel patch to pass P flag info by auxval has launched in Linux-next.

>
> bye,
> //mirabilos
> --
> "Using Lynx is like wearing a really good pair of shades: cuts out
> the glare and harmful UV (ultra-vanity), and you feel so-o-o COOL."
> -- Henry Nelson, March 1999
>
>