Bug#1020802: libz3-4: Xorg crashes on startup due to illegal instruction (SSE2) in libz3-4

[Russ Dill]

Package: libz3-4
Version: 4.8.12-1+b2
Severity: important
X-Debbugs-Cc: russ...@gmail.com

Dear Maintainer,

Running Xorg on my Pentium Pro system crashes with an illegal instruction:

(gdb) run
Starting program: /usr/lib/xorg/Xorg
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".

X.Org X Server 1.21.1.4
X Protocol Version 11, Revision 0
Current Operating System: Linux ppro 5.19.0-2-686 #1 SMP PREEMPT_DYNAMIC Debian 5.19.11-1 (2022-09-24) i686
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.19.0-2-686 root=UUID=dbeee09d-4412-49ab-9c61-a0900452f978 ro apic=debug console=tty0 console=ttyS0,115200 earlyprintk=serial,ttyS0,115200 mitigations=off
xorg-server 2:21.1.4-2 (https://www.debian.org/support)
Current version of pixman: 0.40.0
Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.0.log", Time: Mon Sep 26 15:14:33 2022
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
xf86TokenToOptinfo: table is NULL
xf86TokenToOptinfo: table is NULL

Program received signal SIGILL, Illegal instruction.
0xa95672de in ?? () from /lib/i386-linux-gnu/libz3.so.4

There doesn't seem to be an easy way to build libz3-4 without SSE support.

-- System Information:
Debian Release: bookworm/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: i386 (i686)

Kernel: Linux 5.19.0-2-686 (SMP w/2 CPU threads; PREEMPT)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages libz3-4 depends on:
ii libc6 2.35-1
ii libgcc-s1 12.2.0-3
ii libstdc++6 12.2.0-3

libz3-4 recommends no packages.

libz3-4 suggests no packages.

-- no debconf information

[Bernhard Übelacker]

Dear Maintainer,
tried to have a look at another bug report, but found X not starting up.

I could reproduce this inside a qemu VM started with:
qemu-system-i386 -enable-kvm -cpu pentium-v1 ...

So due to the current documentation [1] it looks like plain Pentium will
not be supported in bookworm, but cannot say where Pentium Pro gets sorted in.

[1] https://www.debian.org/releases/testing/i386/ch02s01.en.html#idm269


Nevertheless, following are a few more details.

Kind regards,
Bernhard


gdb -q --args X :0
(gdb) run
...

Program received signal SIGILL, Illegal instruction.

0xac3672de in ?? () from /lib/i386-linux-gnu/libz3.so.4
(gdb) bt
#0 0xac3672de in ?? () from /lib/i386-linux-gnu/libz3.so.4
#1 0xb7fcdd6b in call_init (env=0xbffffdd0, argv=0xbffffdc4, argc=2, l=<optimized out>) at ./elf/dl-init.c:70
#2 call_init (l=<optimized out>, argc=2, argv=0xbffffdc4, env=0xbffffdd0) at ./elf/dl-init.c:26
#3 0xb7fcde5c in _dl_init (main_map=<optimized out>, argc=2, argv=0xbffffdc4, env=0xbffffdd0) at ./elf/dl-init.c:117
#4 0xb7fd4d97 in call_dl_init (closure=0xbfffe570) at ./elf/dl-open.c:485
#5 0xb7965934 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at ./elf/dl-error-skeleton.c:182
#6 0xb7fd4d25 in dl_open_worker (a=0xbfffe6b8) at ./elf/dl-open.c:808
#7 0xb79658d7 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at ./elf/dl-error-skeleton.c:208
#8 0xb7fd50c0 in _dl_open (file=0xbfffe98c "/usr/lib/i386-linux-gnu/dri/zink_dri.so", mode=-2147483390, caller_dlopen=0xb713a8a5, nsid=<optimized out>, argc=2, argv=0xbffffdc4, env=0xbffffdd0) at ./elf/dl-open.c:886
#9 0xb787f848 in dlopen_doit (a=0xbfffe91c) at ./dlfcn/dlopen.c:56
#10 0xb79658d7 in __GI__dl_catch_exception (exception=<optimized out>, operate=<optimized out>, args=<optimized out>) at ./elf/dl-error-skeleton.c:208
#11 0xb79659a0 in __GI__dl_catch_error (objname=0xbfffe8d4, errstring=0xbfffe8d8, mallocedp=0xbfffe8d3, operate=0xb787f7d0 <dlopen_doit>, args=0xbfffe91c) at ./elf/dl-error-skeleton.c:227
#12 0xb7fe1df8 in _rtld_catch_error (objname=0xbfffe8d4, errstring=0xbfffe8d8, mallocedp=0xbfffe8d3, operate=0xb787f7d0 <dlopen_doit>, args=0xbfffe91c) at ./elf/dl-error-skeleton.c:260
#13 0xb787f297 in _dlerror_run (operate=<optimized out>, args=<optimized out>) at ./dlfcn/dlerror.c:138
#14 0xb787f918 in dlopen_implementation (dl_caller=<optimized out>, mode=258, file=0xbfffe98c "/usr/lib/i386-linux-gnu/dri/zink_dri.so") at ./dlfcn/dlopen.c:71
#15 ___dlopen (file=0xbfffe98c "/usr/lib/i386-linux-gnu/dri/zink_dri.so", mode=258) at ./dlfcn/dlopen.c:81
#16 0xb713a8a5 in ?? () from /lib/i386-linux-gnu/libgbm.so.1
#17 0xb713aa20 in ?? () from /lib/i386-linux-gnu/libgbm.so.1
#18 0xb7139109 in ?? () from /lib/i386-linux-gnu/libgbm.so.1
#19 0xb7139693 in ?? () from /lib/i386-linux-gnu/libgbm.so.1
#20 0xb713995f in ?? () from /lib/i386-linux-gnu/libgbm.so.1
#21 0xb71377cb in ?? () from /lib/i386-linux-gnu/libgbm.so.1
#22 0xb7137935 in gbm_create_device () from /lib/i386-linux-gnu/libgbm.so.1
#23 0xb714c450 in glamor_egl_init () from /usr/lib/xorg/modules/libglamoregl.so
#24 0xb71a2c03 in ?? () from /usr/lib/xorg/modules/drivers/modesetting_drv.so
#25 0x0048aade in InitOutput ()
#26 0x0044a3c9 in ?? ()
#27 0x00432b2b in ?? ()
#28 0xb78213b5 in __libc_start_call_main (main=main@entry=0x432b00, argc=argc@entry=2, argv=argv@entry=0xbffffdc4) at ../sysdeps/nptl/libc_start_call_main.h:58
#29 0xb782147f in __libc_start_main_impl (main=0x432b00, argc=2, argv=0xbffffdc4, init=0x0, fini=0x0, rtld_fini=0xb7fcd930 <_dl_fini>, stack_end=0xbffffdbc) at ../csu/libc-start.c:389
#30 0x00432b67 in _start ()
(gdb) display/i $pc
1: x/i $pc
=> 0xac3672de: pxor %xmm0,%xmm0


With libz3-4-dbgsym:

Core was generated by `/usr/lib/xorg/Xorg :0'.
Program terminated with signal SIGILL, Illegal instruction.
#0 std::__mutex_base::__mutex_base (this=0x6f3fe0) at /usr/include/c++/12/bits/std_mutex.h:65

warning: Source file is more recent than executable.
65 constexpr __mutex_base() noexcept = default;
(gdb) bt 5
#0 std::__mutex_base::__mutex_base (this=0x6f3fe0) at /usr/include/c++/12/bits/std_mutex.h:65
#1 std::mutex::mutex (this=0x6f3fe0) at /usr/include/c++/12/bits/std_mutex.h:91
#2 __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at ./src/util/memory_manager.cpp:39
#3 _GLOBAL__sub_I_memory_manager.cpp(void) () at ./src/util/memory_manager.cpp:373
#4 0xb7fcdd6b in call_init (env=0xbffffdd0, argv=0xbffffdc4, argc=2, l=<optimized out>) at ./elf/dl-init.c:70
(More stack frames follow...)


(Same issue also for kms_swrast_dri.so and swrast_dri.so.)


And came also to this closed upstream issue:
https://github.com/Z3Prover/z3/issues/6369


A workaround might be to move these files out of /usr/lib/i386-linux-gnu/dri:
kms_swrast_dri.so
swrast_dri.so
zink_dri.so
(Or get it some other way not loaded by Xorg.)

[karogyoker999]

Hello,

MR has been opened to "fix" this issue:
https://salsa.debian.org/pkg-llvm-team/z3/-/merge_requests/6

Eliminating SSE2 from z3 upstream is not feasible:
https://github.com/Z3Prover/z3/issues/6369#issuecomment-1259419466

The dependency chain I have identified:
xorg->libgl1-mesa-dri->libllvm14->libz3-4

Regards,
Karo

[Bernhard Übelacker]

Am 10.10.22 um 17:21 schrieb karogyoker999:

Hello Karo,
I am not sure if this the right approach,
because russdill made a few lines below you linked comment
that the crash seems just a product of loading a library,
which does use those instruction in its initialisation step,
even when this library is not suitable for the available hardware.

As far as I see your MR just adds a dependency to sse2-support,
which I guess just makes the installation abort in case of a CPU
not supporting sse2, so I guess this would just make
mesa not installable on your hardware?

Kind regards,
Bernhard

[karogyoker999]

> As far as I see your MR just adds a dependency to sse2-support,
> which I guess just makes the installation abort in case of a CPU
> not supporting sse2, so I guess this would just make
> mesa not installable on your hardware?

Exactly.

It will abort the installation as it wouldn't work anyways.
Mesa now requires SSE2 and therefore Xorg as well.
The workaround is to use Wayland with GNOME or KDE.
But KDE requires SSE2 since several months too (and therefore fails to install).
It is better to have the installation fail rather than
having an unusable system after install. Because in the latter case,
it would mean that users would be disappointed, much time would be wasted
to troubleshoot. Failing the installation with a clear error message
is a much better solution.

The alternative would be to disable SSE2 for all i386 packages of libz3-4.
But usually maintainers don't like that since that would have a performance
impact on Pentium 4 CPUs. So there is a choice to make: disable SSE2 on
all x86_32 CPUs or fail the install on everything except Pentium 4.

There is a Debian patch [1] for the z3 package which does something with SSE2.
But then why is it still crashing? Maybe z3 has some hardcoded SSE2
instruction somewhere?
Or it has been just compiled with the incorrect compiler flags?

To have both performance and compatibility, a runtime check would be needed
to be added, but that would mean a significant amount of work.

Until this work is not done by someone, failing the installation is
still better than
crashing and leaving the system in an unbootable state.

[1]: https://sources.debian.org/patches/z3/4.8.12-1/00-intrinsics.patch/

[Russ Dill]

On Tue, Oct 11, 2022 at 4:06 AM karogyoker999 <karog...@gmail.com> wrote:
>
> > As far as I see your MR just adds a dependency to sse2-support,
> > which I guess just makes the installation abort in case of a CPU
> > not supporting sse2, so I guess this would just make
> > mesa not installable on your hardware?
>
> Exactly.
>
> It will abort the installation as it wouldn't work anyways.
> Mesa now requires SSE2 and therefore Xorg as well.
> The workaround is to use Wayland with GNOME or KDE.
> But KDE requires SSE2 since several months too (and therefore fails to install).
> It is better to have the installation fail rather than
> having an unusable system after install. Because in the latter case,
> it would mean that users would be disappointed, much time would be wasted
> to troubleshoot. Failing the installation with a clear error message
> is a much better solution.

This is a bit of a further pain point for those out there running on
P6 systems as they commonly have video hardware that only supports
DRI1.

> The alternative would be to disable SSE2 for all i386 packages of libz3-4.
> But usually maintainers don't like that since that would have a performance
> impact on Pentium 4 CPUs. So there is a choice to make: disable SSE2 on
> all x86_32 CPUs or fail the install on everything except Pentium 4.

According to the developers of libz4-3, the precision of the FPU x87
registers is not sufficient therefore a libz4-3 package compiled
without SSE2 would be non-functional. It would be far better if
libz3-4 could fail late rather than early. If it could fail later, in
the case that calls to libz3-4 were made I think that would avoid the
current issue. Perhaps if video hardware that requires shader
compilation isn't present, no failures would occur. However the crash
occurs at dlopen time as there are a number of c++ static initializers
that get executed so even if no calls to libz3-4 are even made, or
even calls to the mesa library that links to libllvm14, which links to
libz3-4, everything works fine.

I'm not even certain that libz3-4 is ever even called by mesa. I think
it's just an option feature of libllvm and because mesa links libllvm,
it ends up calling dlopen on the libz3-4 library. The debian changelog
reads:

* Enable Z3 solver (llvm & clang) to improve the quality of the static
analysis results

This doesn't sound like a feature mesa needs and I can't find any
mention of mesa using the z3 solver.

> There is a Debian patch [1] for the z3 package which does something with SSE2.
> But then why is it still crashing? Maybe z3 has some hardcoded SSE2
> instruction somewhere?
> Or it has been just compiled with the incorrect compiler flags?

Doesn't appear to be disabling SSE2, just trying to make something
work when SSE2 is disabled. Not sure why.

> To have both performance and compatibility, a runtime check would be needed
> to be added, but that would mean a significant amount of work.
>
> Until this work is not done by someone, failing the installation is
> still better than
> crashing and leaving the system in an unbootable state.
>
> [1]: https://sources.debian.org/patches/z3/4.8.12-1/00-intrinsics.patch/

The package pulling in libz3-4 into the dependency chain is libllvm14.
I've noticed the ubuntu version of the package does not link against
libz3-4 and the libz3-4 package can be removed from the system and
xorg and mesa function just fine. Perhaps a version of libllvm14
compiled without libz3-4 support that could be installed on instead or
even along side?

[Russ Dill]

> The package pulling in libz3-4 into the dependency chain is libllvm14.
> I've noticed the ubuntu version of the package does not link against
> libz3-4 and the libz3-4 package can be removed from the system and
> xorg and mesa function just fine. Perhaps a version of libllvm14
> compiled without libz3-4 support that could be installed on instead or
> even along side?

A look at the control file for libllvm14 gives a little more detail:

Z3_FLAG = -DLLVM_ENABLE_Z3_SOLVER=OFF
ifeq ($(shell dpkg --compare-versions $(shell dpkg-query -W -f
'$${Version}' libz3-dev) gt 4.7.0; echo $$?),0)
# no ocaml support in main for Ubuntu
ifneq ($(shell dpkg-vendor --derives-from Ubuntu && echo yes),yes)
Z3_FLAG = -DLLVM_ENABLE_Z3_SOLVER=ON
endif
endif
STAGE_2_CMAKE_EXTRA += $(Z3_FLAG)

https://www.cambus.net/clang-static-analyzer-and-the-z3-constraint-solver/

I'm pretty sure the "no ocaml support in main for Ubuntu" is in error
and should read "no z3 constraint solver support in main for Ubuntu".
The author seems to have gotten the similarly named z3 constraint
solver and the LLVM backend for ocaml (Z3) confused.

https://github.com/ohmi/Z3

[karogyoker999]

> According to the developers of libz4-3, the precision of the FPU x87
> registers is not sufficient therefore a libz4-3 package compiled
> without SSE2 would be non-functional.

I think this is just a made-up excuse so they don't have to waste time
on supporting niche 20+ years old hardware.
But it doesn't matter what the real reasons are.
If upstream doesn't want to support a configuration
that is up to them.
What Debian can do is these three things:
1. Use different compiler flags so it will be compiled
without SSE2.
2. If the first approach is not possible (because the
code contains hard-coded unconditional SSE2 assembly code
for example) then patch out these parts of the code.
3. If the second approach requires too much work or
not feasible, then add the sse2-support package as a
dependency.

#1: I'm not sure about that Debian is using the correct
compiler flags when compiling libz3-4 for i386.

For option #2, SIMDEverywhere [1] could be tried.
If replacing emmintrin.h with simde/x86/sse2.h
at [2] solves the issue, that's great.

If both options #1 and #2 fail, then adding the
sse2-support package as a dependency closes this
very issue. To fix the root cause, a new bug
should be opened for the LLVM package to have the
libz3-4 package only as a recommended dependency
(instead of a hard dependency) on i386.

[1]: https://wiki.debian.org/SIMDEverywhere
[2]: https://github.com/Z3Prover/z3/blob/8e0d9bf42def6b89d8ded2dc3652609e8c446183/src/util/hwf.cpp#L56

[karogyoker999]

It seems to me that the libz3-4 package was compiled with SSE2 turned on.

$ objdump -d libz3.so.4 | grep pxor | wc -l
2896

$ objdump -d libz3.so.4 | grep movq | wc -l
26380

[Bernhard Übelacker]

Hello Karo, hello Dill,
I tried to carry this issue to the llvm people, if it
might be possible to drop libz3 linking of libllvm at i386.
This should mitigate most visible issues,
but I am not sure what the downsides would be.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1021926

Kind regards,
Bernhard

[Russ Dill]

I'm pretty happy with that solution. the downside would be that the Z3
constraint solver can't be used by LLVM. Ubuntu and Fedora currently
have it disabled for all architectures and I'm not seeing any bug
reports.

[karogyoker999]

I've opened a new MR:
https://salsa.debian.org/pkg-llvm-team/z3/-/merge_requests/8

This MR won't add SSE2 as a requirement. Instead, it will make
the compiler not to emit SSE2 instructions when building for x86_32.
I've tested it on my Athlon XP and it works fine.
objdump -d doesn't show up any SSE2 instructions anymore.
If I compile the original source of z3, SSE2 will be emitted,
even when I'm building on an Athlon XP.

As a next step, I'll try the SIMDEverywhere approach.

[karogyoker999]

Hello,

My MR [1] using SIMDEverywhere has been merged into z3.
It is going to solve the SSE2 issue on i386 on CPUs
like Athlon XP and Pentium Pro or anything before
Pentium 4.

[1]: https://salsa.debian.org/pkg-llvm-team/z3/-/merge_requests/9