Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bug#1053101: qemu-user-static: segfault when running ldd/arm64 on certain arm64 binaries
320 views
Skip to first unread message
Andreas Beckmann
Sep 27, 2023, 9:10:05 AM9/27/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u1
Severity: important
Control: found -1 1:7.2+dfsg-7+deb12u2
Hi,
on an amd64 host in an arm64 sid chroot which uses qemu-user-static for
executing the foreign binaries, ldd fails on certain binaries:
# file /usr/bin/ls
/usr/bin/ls: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=9f127c37a4c459cf01639f6ded2fcf11a49d3da9, for GNU/Linux 3.7.0, stripped
# ldd /usr/bin/ls
libselinux.so.1 => /lib/aarch64-linux-gnu/libselinux.so.1 (0x00000055028a0000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x00000055028f0000)
/lib/ld-linux-aarch64.so.1 (0x0000005500000000)
libpcre2-8.so.0 => /lib/aarch64-linux-gnu/libpcre2-8.so.0 (0x0000005502ab0000)
# file /usr/bin/aarch64-linux-gnu-g++-13
/usr/bin/aarch64-linux-gnu-g++-13: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=abd2555af6df7d835a23217915452003fadfc423, for GNU/Linux 3.7.0, stripped
# ldd /usr/bin/aarch64-linux-gnu-g++-13
ldd: exited with unknown exit code (139)
The actual failing subcommand run by the ldd script is
# /lib/ld-linux-aarch64.so.1 --verify /usr/bin/aarch64-linux-gnu-g++-13
Segmentation fault
I haven't seen this ldd failure on shared libraries, yet.
I couldn't reproduce it on a porterbox in an arm64 chroot, so this is
likely dependent on the qemu usage.
If I try to get a core dump (inside the arm64 chroot) and load that into
gdb (on the amd64 host), the backtrace looks like an infinite loop of
signal handlers getting called until the stack got exhausted:
#0 0x0000000000401130 in ?? ()
#1 0x0000000000757e19 in g_str_equal ()
#2 0x000000000075691a in g_hash_table_lookup ()
#3 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#4 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#5 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#6 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#7 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#8 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198704, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198706) at ../../linux-user/signal.c:669
#9 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7a4c3a30, puc=0x7ffd7a4c3900) at ../../linux-user/signal.c:839
#10 <signal handler called>
#11 0x0000000000401130 in ?? ()
#12 0x0000000000757e19 in g_str_equal ()
#13 0x000000000075691a in g_hash_table_lookup ()
#14 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#15 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#16 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#17 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#18 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#19 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198704, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198706) at ../../linux-user/signal.c:669
#20 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7a4c4970, puc=0x7ffd7a4c4840) at ../../linux-user/signal.c:839
#21 <signal handler called>
...
#23572 <signal handler called>
#23573 0x0000000000401130 in ?? ()
#23574 0x0000000000757e19 in g_str_equal ()
#23575 0x000000000075691a in g_hash_table_lookup ()
#23576 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#23577 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#23578 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#23579 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#23580 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#23581 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198704, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198706) at ../../linux-user/signal.c:669
#23582 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7acbe2f0, puc=0x7ffd7acbe1c0) at ../../linux-user/signal.c:839
#23583 <signal handler called>
#23584 0x0000000000401130 in ?? ()
#23585 0x0000000000757e19 in g_str_equal ()
#23586 0x000000000075691a in g_hash_table_lookup ()
#23587 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#23588 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#23589 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#23590 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#23591 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#23592 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198528, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198530) at ../../linux-user/signal.c:669
#23593 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7acbf230, puc=0x7ffd7acbf100) at ../../linux-user/signal.c:839
#23594 <signal handler called>
#23595 0x0000000000401080 in ?? ()
#23596 0x000000000075edb1 in g_malloc0 ()
#23597 0x000000000061abd3 in page_find_alloc (index=index@entry=1024, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:443
#23598 0x000000000061c698 in page_reset_target_data (start=start@entry=4194304, end=end@entry=5128192) at ../../accel/tcg/user-exec.c:233
#23599 0x000000000061b795 in page_set_flags (start=4194304, end=5128192, flags=2125) at ../../accel/tcg/translate-all.c:1395
#23600 0x0000000000627727 in target_mmap (start=start@entry=4194304, len=<optimized out>, len@entry=933888, target_prot=target_prot@entry=5, flags=<optimized out>, fd=fd@entry=3, offset=offset@entry=0) at ../../linux-user/mmap.c:648
#23601 0x0000000000640396 in do_syscall1 (cpu_env=cpu_env@entry=0x2a724e0, num=num@entry=222, arg1=arg1@entry=4194304, arg2=arg2@entry=933888, arg3=arg3@entry=5, arg4=arg4@entry=2066, arg5=<optimized out>, arg6=<optimized out>, arg8=0, arg7=0)
at ../../linux-user/syscall.c:10091
#23602 0x0000000000642895 in do_syscall (cpu_env=cpu_env@entry=0x2a724e0, num=222, arg1=4194304, arg2=933888, arg3=5, arg4=2066, arg5=3, arg6=0, arg7=0, arg8=0) at ../../linux-user/syscall.c:13375
#23603 0x00000000004059af in cpu_loop (env=env@entry=0x2a724e0) at ../../linux-user/aarch64/cpu_loop.c:101
#23604 0x000000000040220c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../../linux-user/main.c:950
(backtrace from 1:7.2+dfsg-7+deb12u2)
Andreas
Version: 1:7.2+dfsg-7+deb12u1
Severity: important
Control: found -1 1:7.2+dfsg-7+deb12u2
Hi,
on an amd64 host in an arm64 sid chroot which uses qemu-user-static for
executing the foreign binaries, ldd fails on certain binaries:
# file /usr/bin/ls
/usr/bin/ls: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=9f127c37a4c459cf01639f6ded2fcf11a49d3da9, for GNU/Linux 3.7.0, stripped
# ldd /usr/bin/ls
libselinux.so.1 => /lib/aarch64-linux-gnu/libselinux.so.1 (0x00000055028a0000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x00000055028f0000)
/lib/ld-linux-aarch64.so.1 (0x0000005500000000)
libpcre2-8.so.0 => /lib/aarch64-linux-gnu/libpcre2-8.so.0 (0x0000005502ab0000)
# file /usr/bin/aarch64-linux-gnu-g++-13
/usr/bin/aarch64-linux-gnu-g++-13: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=abd2555af6df7d835a23217915452003fadfc423, for GNU/Linux 3.7.0, stripped
# ldd /usr/bin/aarch64-linux-gnu-g++-13
ldd: exited with unknown exit code (139)
The actual failing subcommand run by the ldd script is
# /lib/ld-linux-aarch64.so.1 --verify /usr/bin/aarch64-linux-gnu-g++-13
Segmentation fault
I haven't seen this ldd failure on shared libraries, yet.
I couldn't reproduce it on a porterbox in an arm64 chroot, so this is
likely dependent on the qemu usage.
If I try to get a core dump (inside the arm64 chroot) and load that into
gdb (on the amd64 host), the backtrace looks like an infinite loop of
signal handlers getting called until the stack got exhausted:
#0 0x0000000000401130 in ?? ()
#1 0x0000000000757e19 in g_str_equal ()
#2 0x000000000075691a in g_hash_table_lookup ()
#3 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#4 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#5 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#6 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#7 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#8 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198704, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198706) at ../../linux-user/signal.c:669
#9 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7a4c3a30, puc=0x7ffd7a4c3900) at ../../linux-user/signal.c:839
#10 <signal handler called>
#11 0x0000000000401130 in ?? ()
#12 0x0000000000757e19 in g_str_equal ()
#13 0x000000000075691a in g_hash_table_lookup ()
#14 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#15 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#16 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#17 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#18 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#19 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198704, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198706) at ../../linux-user/signal.c:669
#20 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7a4c4970, puc=0x7ffd7a4c4840) at ../../linux-user/signal.c:839
#21 <signal handler called>
...
#23572 <signal handler called>
#23573 0x0000000000401130 in ?? ()
#23574 0x0000000000757e19 in g_str_equal ()
#23575 0x000000000075691a in g_hash_table_lookup ()
#23576 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#23577 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#23578 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#23579 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#23580 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#23581 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198704, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198706) at ../../linux-user/signal.c:669
#23582 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7acbe2f0, puc=0x7ffd7acbe1c0) at ../../linux-user/signal.c:839
#23583 <signal handler called>
#23584 0x0000000000401130 in ?? ()
#23585 0x0000000000757e19 in g_str_equal ()
#23586 0x000000000075691a in g_hash_table_lookup ()
#23587 0x000000000064c36b in type_table_lookup (name=0x8868fe "cpu") at ../../qom/object.c:87
#23588 type_get_by_name (name=0x8868fe "cpu") at ../../qom/object.c:175
#23589 object_class_dynamic_cast (class=class@entry=0x2a5da40, typename=typename@entry=0x8868fe "cpu") at ../../qom/object.c:925
#23590 0x000000000064c572 in object_class_dynamic_cast_assert (class=0x2a5da40, typename=typename@entry=0x8868fe "cpu", file=file@entry=0x86a01d "./include/hw/core/cpu.h", line=line@entry=62, func=func@entry=0x8c3490 <__func__.0> "CPU_GET_CLASS")
at ../../qom/object.c:981
#23591 0x000000000062850a in CPU_GET_CLASS (obj=obj@entry=0x2a721c0) at ./include/hw/core/cpu.h:62
#23592 0x00000000006298fc in cpu_loop_exit_sigsegv (cpu=cpu@entry=0x2a721c0, addr=addr@entry=4198528, access_type=access_type@entry=MMU_DATA_LOAD, maperr=maperr@entry=true, ra=4198530) at ../../linux-user/signal.c:669
#23593 0x0000000000629dda in host_signal_handler (host_sig=11, info=0x7ffd7acbf230, puc=0x7ffd7acbf100) at ../../linux-user/signal.c:839
#23594 <signal handler called>
#23595 0x0000000000401080 in ?? ()
#23596 0x000000000075edb1 in g_malloc0 ()
#23597 0x000000000061abd3 in page_find_alloc (index=index@entry=1024, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:443
#23598 0x000000000061c698 in page_reset_target_data (start=start@entry=4194304, end=end@entry=5128192) at ../../accel/tcg/user-exec.c:233
#23599 0x000000000061b795 in page_set_flags (start=4194304, end=5128192, flags=2125) at ../../accel/tcg/translate-all.c:1395
#23600 0x0000000000627727 in target_mmap (start=start@entry=4194304, len=<optimized out>, len@entry=933888, target_prot=target_prot@entry=5, flags=<optimized out>, fd=fd@entry=3, offset=offset@entry=0) at ../../linux-user/mmap.c:648
#23601 0x0000000000640396 in do_syscall1 (cpu_env=cpu_env@entry=0x2a724e0, num=num@entry=222, arg1=arg1@entry=4194304, arg2=arg2@entry=933888, arg3=arg3@entry=5, arg4=arg4@entry=2066, arg5=<optimized out>, arg6=<optimized out>, arg8=0, arg7=0)
at ../../linux-user/syscall.c:10091
#23602 0x0000000000642895 in do_syscall (cpu_env=cpu_env@entry=0x2a724e0, num=222, arg1=4194304, arg2=933888, arg3=5, arg4=2066, arg5=3, arg6=0, arg7=0, arg8=0) at ../../linux-user/syscall.c:13375
#23603 0x00000000004059af in cpu_loop (env=env@entry=0x2a724e0) at ../../linux-user/aarch64/cpu_loop.c:101
#23604 0x000000000040220c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at ../../linux-user/main.c:950
(backtrace from 1:7.2+dfsg-7+deb12u2)
Andreas
Michael Tokarev
Sep 27, 2023, 9:30:05 AM9/27/23
to
27.09.2023 16:04, Andreas Beckmann wrote:
> Package: qemu-user-static
> Version: 1:7.2+dfsg-7+deb12u1
> Severity: important
> Control: found -1 1:7.2+dfsg-7+deb12u2
Hmm wait. Are you trying to say this happens on deb12u2
> Package: qemu-user-static
> Version: 1:7.2+dfsg-7+deb12u1
> Severity: important
> Control: found -1 1:7.2+dfsg-7+deb12u2
but does NOT happen on deb12u1?
/mjt
Andreas Beckmann
Sep 27, 2023, 9:30:05 AM9/27/23
to
On 27/09/2023 15.16, Michael Tokarev wrote:
>> on an amd64 host in an arm64 sid chroot which uses qemu-user-static for
>> executing the foreign binaries, ldd fails on certain binaries:
>
> This most likely is https://gitlab.com/qemu-project/qemu/-/issues/1763
>> on an amd64 host in an arm64 sid chroot which uses qemu-user-static for
>> executing the foreign binaries, ldd fails on certain binaries:
>
That could be related, except that in my case ldd /usr/bin/ls is working
while ldd /usr/bin/g++ is failing ...
Andreas
Michael Tokarev
Sep 27, 2023, 9:30:05 AM9/27/23
to
27.09.2023 16:04, Andreas Beckmann:
> Package: qemu-user-static
> Version: 1:7.2+dfsg-7+deb12u1
> Severity: important
> Control: found -1 1:7.2+dfsg-7+deb12u2
>
> Hi,
>
> on an amd64 host in an arm64 sid chroot which uses qemu-user-static for
> executing the foreign binaries, ldd fails on certain binaries:
/mjt
> Version: 1:7.2+dfsg-7+deb12u1
> Severity: important
> Control: found -1 1:7.2+dfsg-7+deb12u2
>
> Hi,
>
> on an amd64 host in an arm64 sid chroot which uses qemu-user-static for
> executing the foreign binaries, ldd fails on certain binaries:
Andreas Beckmann
Sep 27, 2023, 9:40:04 AM9/27/23
to
On 27/09/2023 15.23, Michael Tokarev wrote:
> Hmm wait. Are you trying to say this happens on deb12u2
> but does NOT happen on deb12u1?
No. It happens on both. I observed it on u1 then upgraded to u2 in case
> Hmm wait. Are you trying to say this happens on deb12u2
> but does NOT happen on deb12u1?
it could be fixed there, but nothing changed.
(I haven't tried 8.x, yet, since last time I tried 8.x didn't work at
all in my setup and I didn't have time to debug that yet.)
andreas
Michael Tokarev
Sep 27, 2023, 12:20:05 PM9/27/23
to
27.09.2023 16:29, Andreas Beckmann wrote:
> On 27/09/2023 15.23, Michael Tokarev wrote:
>> Hmm wait. Are you trying to say this happens on deb12u2
>> but does NOT happen on deb12u1?
>
> No. It happens on both. I observed it on u1 then upgraded to u2 in case it could be fixed there, but nothing changed.
Ok. I was nervous thinking it's a regression in 7.2.x stable series.
> On 27/09/2023 15.23, Michael Tokarev wrote:
>> Hmm wait. Are you trying to say this happens on deb12u2
>> but does NOT happen on deb12u1?
>
> No. It happens on both. I observed it on u1 then upgraded to u2 in case it could be fixed there, but nothing changed.
> (I haven't tried 8.x, yet, since last time I tried 8.x didn't work at all in my setup and I didn't have time to debug that yet.)
issue known to date. Some of that changes went to 8.0.x stable too,
but most important didn't. Debian testing now - finally - has 8.1.
So it is definitely worth to try this one.
I'd love to have a (simple) reproducer for this one..
/mjt
Michael Tokarev
Sep 28, 2023, 1:50:05 AM9/28/23
to
Control: tag -1 + confirmed
Control: found -1 1:8.1.0+ds-6
27.09.2023 16:04, Andreas Beckmann:
And the same issue is also present in 8.1.0 and 8.1.1. Sigh.
/mjt
Control: found -1 1:8.1.0+ds-6
27.09.2023 16:04, Andreas Beckmann:
> # /lib/ld-linux-aarch64.so.1 --verify /usr/bin/aarch64-linux-gnu-g++-13
> Segmentation fault
Ok, I can reproduce this one, which is good!
> Segmentation fault
And the same issue is also present in 8.1.0 and 8.1.1. Sigh.
/mjt
Michael Tokarev
Sep 28, 2023, 4:20:04 AM9/28/23
to
28.09.2023 08:39, Michael Tokarev wrote:
> 27.09.2023 16:04, Andreas Beckmann:
>
>> # /lib/ld-linux-aarch64.so.1 --verify /usr/bin/aarch64-linux-gnu-g++-13
>> Segmentation fault
>
> Ok, I can reproduce this one, which is good!
> And the same issue is also present in 8.1.0 and 8.1.1. Sigh.
Now, this is interesting, -- the issue is caused by --disable-pie flag which
> 27.09.2023 16:04, Andreas Beckmann:
>
>> # /lib/ld-linux-aarch64.so.1 --verify /usr/bin/aarch64-linux-gnu-g++-13
>> Segmentation fault
>
> Ok, I can reproduce this one, which is good!
> And the same issue is also present in 8.1.0 and 8.1.1. Sigh.
is being used to build qemu-user-static on Debian. Without --disable-pie, the
resulting executable just works without exposing this issue. Regular qemu-user
works fine too.
Hmm..
/mjt
Tj
Nov 24, 2023, 11:40:05 AM11/24/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
Initially I noticed the ELF types shown for ls (0 a.k.a. SYSV) and aarch64-linux-gnu-g++-13 (3 a.k.a. GNU/Linux) are different.
This is octet offset 7 of the ELF header, e_ident[EI_OSABI]. When it is 3 the following field (offset 8) e_ident[EI_ABIVERSION] contains
the ABI version of the dynamic linker.
Since the issue seems related to some interaction with static vs dynamic this may be a clue!
Doing some debugging on amd64 host with an aarch64 sid chroot build we get a
clearer idea of what is going wrong. It looks like in
accel/tcg/translate-all.c::page_find_alloc()
`void **lp` is the problem but has been optimised out by the compiler so I'll need to build
qemu without optimisations and retry.
$ gdb --directory /srv/NAS/Sunny/SourceCode/qemu/qemu-7.2+dfsg/tcg/aarch64 --args /usr/libexec/qemu-binfmt/aarch64-binfmt-P sid-aarch64/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
Reading symbols from /usr/libexec/qemu-binfmt/aarch64-binfmt-P...
Reading symbols from /usr/lib/debug/.build-id/7b/ef74adf0c2ded7f731079d47cc8ca9dcc579e1.debug...
(gdb) start
Temporary breakpoint 1 at 0x401b90: file ../../linux-user/main.c, line 664.
Starting program: /usr/libexec/qemu-binfmt/aarch64-binfmt-P sid-aarch64/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7ff86c0 (LWP 250079)]
Thread 1 "aarch64-binfmt-" hit Temporary breakpoint 1, main (argc=4, argv=0x7fffffffdb68, envp=0x7fffffffdb90) at ../../linux-user/main.c:664
664 {
(gdb) break page_find_alloc
(gdb) break do_syscall1 if num == 222
(gdb) c
Thread 1 "aarch64-binfmt-" hit Breakpoint 6, page_find_alloc (index=index@entry=89128960, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:431
431 lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
(gdb) info break
Num Type Disp Enb Address What
6 breakpoint keep y 0x000000000061ab70 in page_find_alloc at ../../accel/tcg/translate-all.c:431
breakpoint already hit 1 time
8 breakpoint keep y 0x000000000063a660 in do_syscall1 at ../../linux-user/syscall.c:8615
stop only if num == 222
(gdb) disable 6
(gdb) c
Continuing.
Thread 1 "aarch64-binfmt-" hit Breakpoint 8, do_syscall1 (cpu_env=cpu_env@entry=0xe44220, num=num@entry=222, arg1=arg1@entry=4194304, arg2=arg2@entry=937984,
arg3=arg3@entry=5, arg4=arg4@entry=2066, arg5=3, arg6=0, arg8=<optimized out>, arg7=<optimized out>) at ../../linux-user/syscall.c:8615
8615 static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
(gdb) enable 6
(gdb) c
Continuing.
Thread 1 "aarch64-binfmt-" hit Breakpoint 6, page_find_alloc (index=index@entry=1024, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:431
431 lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
(gdb) p lp
$1 = <optimized out>
(gdb) p l1_map
$5 = {0xe9e3f0, 0x0 <repeats 8191 times>}
(gdb) p v_l1_shift
$6 = 40
(gdb) p v_l1_size
$7 = 4096
(gdb) p index
$8 = 1024
(gdb) p ((index >> v_l1_shift))
$9 = 0
(gdb) p ((index >> v_l1_shift) & (v_l1_size -1))
$10 = 0
(gdb) p l1_map + ((index >> v_l1_shift) & (v_l1_size -1))
$11 = (void **) 0xe04580 <l1_map>
(gdb) n
Thread 1 "aarch64-binfmt-" received signal SIGSEGV, Segmentation fault.
page_find_alloc (index=index@entry=1024, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:431
431 lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
(gdb) p lp
$12 = <optimized out>
(gdb) n
host_signal_handler (host_sig=11, info=0x7fffffffb8f0, puc=0x7fffffffb7c0) at ../../linux-user/signal.c:783
783 {
(gdb) n
793 void *sigmask = host_signal_mask(uc);
(gdb) n
784 CPUArchState *env = thread_cpu->env_ptr;
(gdb)
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
Initially I noticed the ELF types shown for ls (0 a.k.a. SYSV) and aarch64-linux-gnu-g++-13 (3 a.k.a. GNU/Linux) are different.
This is octet offset 7 of the ELF header, e_ident[EI_OSABI]. When it is 3 the following field (offset 8) e_ident[EI_ABIVERSION] contains
the ABI version of the dynamic linker.
Since the issue seems related to some interaction with static vs dynamic this may be a clue!
Doing some debugging on amd64 host with an aarch64 sid chroot build we get a
clearer idea of what is going wrong. It looks like in
accel/tcg/translate-all.c::page_find_alloc()
`void **lp` is the problem but has been optimised out by the compiler so I'll need to build
qemu without optimisations and retry.
$ gdb --directory /srv/NAS/Sunny/SourceCode/qemu/qemu-7.2+dfsg/tcg/aarch64 --args /usr/libexec/qemu-binfmt/aarch64-binfmt-P sid-aarch64/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
Reading symbols from /usr/libexec/qemu-binfmt/aarch64-binfmt-P...
Reading symbols from /usr/lib/debug/.build-id/7b/ef74adf0c2ded7f731079d47cc8ca9dcc579e1.debug...
(gdb) start
Temporary breakpoint 1 at 0x401b90: file ../../linux-user/main.c, line 664.
Starting program: /usr/libexec/qemu-binfmt/aarch64-binfmt-P sid-aarch64/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7ff86c0 (LWP 250079)]
Thread 1 "aarch64-binfmt-" hit Temporary breakpoint 1, main (argc=4, argv=0x7fffffffdb68, envp=0x7fffffffdb90) at ../../linux-user/main.c:664
664 {
(gdb) break page_find_alloc
(gdb) break do_syscall1 if num == 222
(gdb) c
Thread 1 "aarch64-binfmt-" hit Breakpoint 6, page_find_alloc (index=index@entry=89128960, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:431
431 lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
(gdb) info break
Num Type Disp Enb Address What
6 breakpoint keep y 0x000000000061ab70 in page_find_alloc at ../../accel/tcg/translate-all.c:431
breakpoint already hit 1 time
8 breakpoint keep y 0x000000000063a660 in do_syscall1 at ../../linux-user/syscall.c:8615
stop only if num == 222
(gdb) disable 6
(gdb) c
Continuing.
Thread 1 "aarch64-binfmt-" hit Breakpoint 8, do_syscall1 (cpu_env=cpu_env@entry=0xe44220, num=num@entry=222, arg1=arg1@entry=4194304, arg2=arg2@entry=937984,
arg3=arg3@entry=5, arg4=arg4@entry=2066, arg5=3, arg6=0, arg8=<optimized out>, arg7=<optimized out>) at ../../linux-user/syscall.c:8615
8615 static abi_long do_syscall1(CPUArchState *cpu_env, int num, abi_long arg1,
(gdb) enable 6
(gdb) c
Continuing.
Thread 1 "aarch64-binfmt-" hit Breakpoint 6, page_find_alloc (index=index@entry=1024, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:431
431 lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
(gdb) p lp
$1 = <optimized out>
(gdb) p l1_map
$5 = {0xe9e3f0, 0x0 <repeats 8191 times>}
(gdb) p v_l1_shift
$6 = 40
(gdb) p v_l1_size
$7 = 4096
(gdb) p index
$8 = 1024
(gdb) p ((index >> v_l1_shift))
$9 = 0
(gdb) p ((index >> v_l1_shift) & (v_l1_size -1))
$10 = 0
(gdb) p l1_map + ((index >> v_l1_shift) & (v_l1_size -1))
$11 = (void **) 0xe04580 <l1_map>
(gdb) n
Thread 1 "aarch64-binfmt-" received signal SIGSEGV, Segmentation fault.
page_find_alloc (index=index@entry=1024, alloc=alloc@entry=true) at ../../accel/tcg/translate-all.c:431
431 lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
(gdb) p lp
$12 = <optimized out>
(gdb) n
host_signal_handler (host_sig=11, info=0x7fffffffb8f0, puc=0x7fffffffb7c0) at ../../linux-user/signal.c:783
783 {
(gdb) n
793 void *sigmask = host_signal_mask(uc);
(gdb) n
784 CPUArchState *env = thread_cpu->env_ptr;
(gdb)
Tj
Nov 24, 2023, 1:40:05 PM11/24/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
I forgot to note that Michael's observation that building
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
qemu-user-static without --disable-pie might reveal a conflict
between the emulator and target linking:
# file /usr/bin/ls
/usr/bin/ls: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=6cc2da196ef5482b32870624563242ff38023843, for GNU/Linux 3.7.0, stripped
# file /usr/bin/aarch64-linux-gnu-g++-13
/usr/bin/aarch64-linux-gnu-g++-13: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=30e11cf9cf6aec0c01374a11a36b1687195905cb, for GNU/Linux 3.7.0, stripped
# objdump -f /usr/bin/ls
/usr/bin/ls: file format elf64-littleaarch64
architecture: aarch64, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x0000000000005cc0
# objdump -f /usr/bin/aarch64-linux-gnu-g++-13
/usr/bin/aarch64-linux-gnu-g++-13: file format elf64-littleaarch64
architecture: aarch64, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x000000000042e3c0
So `ls` is a "pie executable" and does not have "EXEC_P" where g++ is
just "executable" and has "EXEC_P".
The interpreter has:
# file /lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=65beabffba8dc9cab560b4338b8b0d568d21c964, stripped
# objdump -f /lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1: file format elf64-littleaarch64
architecture: aarch64, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x000000000001a1c0
Tj
Nov 24, 2023, 4:50:05 PM11/24/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
Debugging across an architecture boundary when the architecture is
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
emulated is... painful!
However, as I dig in from both sides this is pointing to an issue with
PIE handling. E.g.:
$ qemu-aarch64-static -strace sid-aarch64/usr/lib/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
482357 brk(NULL) = 0x0000005500042000
482357 openat(AT_FDCWD,"sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13",O_RDONLY|O_CLOEXEC) = 3
482357 read(3,0x2841280,832) = 832
482357 getcwd(0x5500041980,128) = 18
482357 mmap(0x0000000000400000,937984,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0)Segmentation fault (core dumped)
$ qemu-aarch64-static -strace sid-aarch64/usr/lib/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/ls
493812 brk(NULL) = 0x0000005500042000
493812 openat(AT_FDCWD,"sid-aarch64/usr/bin/ls",O_RDONLY|O_CLOEXEC) = 3
493812 read(3,0x28412a0,832) = 832
493812 getcwd(0x5500041970,128) = 18
493812 mmap(NULL,334096,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005502844000
493812 mmap(0x0000005502850000,268560,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0) = 0x0000005502850000
493812 munmap(0x0000005502844000,49152) = 0
493812 munmap(0x0000005502892000,14608) = 0
493812 mprotect(0x0000005502873000,114688,PROT_NONE) = 0
493812 mmap(0x000000550288f000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x2f000) = 0x000000550288f000
493812 mmap(0x0000005502891000,2320,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x0000005502891000
493812 close(3) = 0
493812 exit_group(0)
Take a close look at the address of the mmap()s: 0x0000000000400000 vs 0x0000005502850000 - this hints at
the difference handling of PIE and non-PIE.
The latter (for the PIE executable `ls`) does an initial `mmap(NULL, length_of_binary+64k), ...`
This looks like it may be a sign of PIE and Address Space Layout Randomisation
getting confused in the QEMU code between the qemu binary itself and the
target it is interpretting for.
For a GCC binary that is PIE it works:
file /usr/bin/aarch64-linux-gnu-ld.gold
/usr/bin/aarch64-linux-gnu-ld.gold: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=21bf827bc36e1b6cd7d8b280d5b5d385b4946d17, for GNU/Linux 3.7.0, stripped
$ qemu-aarch64-static -strace sid-aarch64/usr/lib/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-ld.gold
505564 brk(NULL) = 0x0000005500042000
505564 openat(AT_FDCWD,"sid-aarch64/usr/bin/aarch64-linux-gnu-ld.gold",O_RDONLY|O_CLOEXEC) = 3
505564 read(3,0x2841280,832) = 832
505564 getcwd(0x5500041980,128) = 18
505564 mmap(NULL,5765104,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005502844000
505564 mmap(0x0000005502850000,5699568,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0) = 0x0000005502850000
505564 munmap(0x0000005502844000,49152) = 0
505564 munmap(0x0000005502dc0000,14320) = 0
505564 mprotect(0x0000005502d72000,73728,PROT_NONE) = 0
505564 mmap(0x0000005502d84000,184320,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x524000) = 0x0000005502d84000
505564 mmap(0x0000005502db1000,59376,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x0000005502db1000
505564 close(3) = 0
505564 exit_group(0)
In the qemu code in `./accel/tcg/translate-all.c::page_find_alloc()`
there are calls to the built-in glib-2.0 g_new0() on the host. Maybe PIE
affects how the glib functions form the underlying mmap() call?
Tj
Nov 25, 2023, 7:30:06 AM11/25/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
Sharing some further in-depth debugging results. Everything seems to
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
point to the executables with ELF type 3 (Linux) not marked as PIE
suffering the same fate. The address that faults is always 0x400000 so
I'm hunting for that in glibc's ld.so since I suspect it is a hard-coded
default when not asking the kernel to suggest an address with
mmap(NULL, ...) that ends up with an address that matches the host
process virtual addresses.
$ /usr/bin/qemu-aarch64-static -d page,strace,cpu,exec,trace:guest_user_syscall,trace:target_mmap sid-aarch64/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
host mmap_min_addr=0x10000
Locating guest address space @ 0x0
target_mmap start=0x0 len=0x2041360 prot=0x0 flags=0x4022 fd=-1 offset=0x0
page layout changed following mmap
start end size prot
0000005500000000-0000005502042000 0000000002042000 ---
target_mmap start=0x5500000000 len=0x26000 prot=0x5 flags=0x12 fd=3 offset=0x0
...
Trace 0: 0x7f4f18029a00 [0000000001009331/00000055000066ac/00000001/00000000]
PC=00000055000066ac X00=0000000000400000 X01=0000000000504968
X02=0000000000504968 X03=0000000000000000 X04=000000006474e551
X05=0000000000001730 X06=0000005502841270 X07=0000000000000006
X08=0000000000006098 X09=0000000000000000 X10=0000000000000002
X11=0000000000000000 X12=00000000004fe8d0 X13=0000000000001000
X14=0000000000010000 X15=000000000000ffff X16=0000000000010000
X17=fffffffffffff000 X18=00000000004fe000 X19=0000000020000000
X20=0000000000000003 X21=0000000000000000 X22=0000005500041390
X23=0000005500041360 X24=0000005502840de0 X25=0000000000104968
X26=0000005502840de0 X27=000000550003f000 X28=00000055028412b0
X29=0000005502841010 X30=0000000000000000 SP=0000005502840db0
PSTATE=80000000 N--- EL0t SVCR=00000000 -- BTYPE=0
Linking TBs 0x7f4f18029a00 index 0 -> 0x7f4f18029bc0
Trace 0: 0x7f4f18029bc0 [0000000001009331/00000055000066c0/00000001/00000000]
PC=00000055000066c0 X00=0000005502840e50 X01=0000000000504968
X02=0000000000504968 X03=0000000000000000 X04=000000006474e551
X05=0000000000001730 X06=0000005502841270 X07=0000000000000006
X08=0000000000006098 X09=0000000000000000 X10=0000000000000002
X11=0000000000000000 X12=00000000004fe8d0 X13=0000000000001000
X14=0000000000010000 X15=000000000000ffff X16=0000000000010000
X17=fffffffffffff000 X18=00000000004fe000 X19=0000000020000000
X20=0000000000000003 X21=0000000000000000 X22=0000005500041390
X23=0000005500041360 X24=0000005502840de0 X25=0000000000104968
X26=0000005502840de0 X27=000000550003f000 X28=00000055028412b0
X29=0000005502841010 X30=0000000000000000 SP=0000005502840db0
PSTATE=80000000 N--- EL0t SVCR=00000000 -- BTYPE=0
Linking TBs 0x7f4f18029bc0 index 0 -> 0x7f4f18029d80
Trace 0: 0x7f4f18029d80 [0000000001009331/00000055000066cc/00000001/00000000]
PC=00000055000066cc X00=0000000000400000 X01=00000000004e5000
X02=0000000000504968 X03=0000000000000000 X04=000000006474e551
X05=0000000000001730 X06=0000005502841270 X07=0000000000000006
X08=0000000000006098 X09=0000000000000000 X10=0000000000000002
X11=0000000000000000 X12=00000000004fe8d0 X13=0000000000001000
X14=0000000000010000 X15=000000000000ffff X16=0000000000010000
X17=fffffffffffff000 X18=00000000004fe000 X19=0000000020000000
X20=0000000000000003 X21=0000000000000000 X22=0000005500041390
X23=0000005500041360 X24=0000005502840de0 X25=0000000000104968
X26=0000005502840de0 X27=000000550003f000 X28=00000055028412b0
X29=0000005502841010 X30=0000000000000000 SP=0000005502840db0
PSTATE=20000000 --C- EL0t SVCR=00000000 -- BTYPE=0
Trace 0: 0x7f4f18029fc0 [0000000001009331/000000550001ab90/00000001/00000000]
PC=000000550001ab90 X00=0000000000400000 X01=00000000000e5000
X02=0000000000000005 X03=0000000000000812 X04=0000000000000003
X05=0000000000000000 X06=0000005502841270 X07=0000000000000006
X08=0000000000006098 X09=0000000000000000 X10=0000000000000002
X11=0000000000000000 X12=00000000004fe8d0 X13=0000000000001000
X14=0000000000010000 X15=000000000000ffff X16=0000000000010000
X17=fffffffffffff000 X18=00000000004fe000 X19=0000000020000000
X20=0000000000000003 X21=0000000000000000 X22=0000005500041390
X23=0000005500041360 X24=0000005502840de0 X25=0000000000104968
X26=0000005502840de0 X27=000000550003f000 X28=00000055028412b0
X29=0000005502841010 X30=00000055000066f4 SP=0000005502840db0
PSTATE=20000000 --C- EL0t SVCR=00000000 -- BTYPE=0
Linking TBs 0x7f4f18029fc0 index 0 -> 0x7f4f1802a140
Trace 0: 0x7f4f1802a140 [0000000001009331/000000550001ab98/00000001/00000000]
PC=000000550001ab98 X00=0000000000400000 X01=00000000000e5000
X02=0000000000000005 X03=0000000000000812 X04=0000000000000003
X05=0000000000000000 X06=0000005502841270 X07=0000000000000006
X08=0000000000006098 X09=0000000000000000 X10=0000000000000002
X11=0000000000000000 X12=00000000004fe8d0 X13=0000000000001000
X14=0000000000010000 X15=000000000000ffff X16=0000000000010000
X17=fffffffffffff000 X18=00000000004fe000 X19=0000000020000000
X20=0000000000000003 X21=0000000000000000 X22=0000005500041390
X23=0000005500041360 X24=0000005502840de0 X25=0000000000104968
X26=0000005502840de0 X27=000000550003f000 X28=00000055028412b0
X29=0000005502841010 X30=00000055000066f4 SP=0000005502840db0
PSTATE=40000000 -Z-- EL0t SVCR=00000000 -- BTYPE=0
guest_user_syscall cpu=0x1659e50 num=0x00000000000000de arg1=0x0000000000400000 arg2=0x00000000000e5000 arg3=0x0000000000000005 arg4=0x0000000000000812 arg5=0x000000000000
0003 arg6=0x0000000000000000 arg7=0x0000000000000000 arg8=0x0000000000000000
1084043 mmap(0x0000000000400000,937984,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0)target_mmap start=0x400000 len=0xe5000 prot=0x5 flags=0x812 fd=3 offset=
0x0
Segmentation fault (core dumped)
Notice how the failing mmap address is 0x00400000 but the emulator
mapping is at 0x5500000000.
My suspicion here is that some kind of address translation should be
done in QEMU runtime code based on the permutations of PIE between host
and target.
I discounted ASLR being the cause via
$ cat /proc/sys/kernel/randomize_va_space
2
$ echo 0 | sudo dd of=/proc/sys/kernel/randomize_va_space
Tj
Nov 25, 2023, 10:30:05 AM11/25/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
After building without `--disable-pie` and observing differences and
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
results I suspect the cause is something that is so obvious I missed it!
The emulator - when totally static - maps to 0x00040000 itself. Then in
the target ld-linux-aarch64.so.1 tries to map a static aarch64
executable to the same address (0x00040000) and since qemu doesn't
translate that in any way (could it?) the SIGSEGV occurs because the
emulator (qemu-aarch64-static) already uses that address.
However... the mmap() documentation suggests it should return -1 and set
errno and I'd expect that to be EEXIST MAP_FIXED_NOREPLACE.
However #2... those same docs state SIGSEGV when "Attempted write into a
region mapped as read-only" and that does make sense if the emulator's
.text is mapped there (presumably as PROT_EXEC|PROT_READ and
MAP_DENYWRITE).
Tj
Nov 26, 2023, 5:00:06 AM11/26/23
to
Package: qemu-user-static
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
After several experiments and using `readelf -l` to check that there is
Version: 1:7.2+dfsg-7+deb12u2
Followup-For: Bug #1053101
no INTERP Program Header and testing with
qemu-7.2+dfsg/b/user-static/qemu-aarch64 -d page sid-aarch64/lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
I am reasonably confident that removing --disable-pie is the solution
for this.
The only caveat is the tests are specific to amd64 host and aarch64
target and I cannot test all the other potential combinations for a
regression.
However I've included a debdiff with the proposed change.
Michael Tokarev
Nov 26, 2023, 7:30:05 AM11/26/23
to
particular issue, is not a solution, it is a workaround for this particular
platform combination. At the very least, only platforms where -static-pie
works to begin with, are "fixed" this way.
> However I've included a debdiff with the proposed change.
fwiw :)
Also as mentioned on IRC, I re-enabled -static-pie in experimental first
(and immediately faced an issue, - it doesn't work on i386, #1056739).
/mjt
Richard Henderson
Nov 30, 2023, 2:10:07 PM11/30/23
to
On Fri, 24 Nov 2023 21:43:54 +0000 Tj <deb...@iam.tj> wrote:
> $ qemu-aarch64-static -strace sid-aarch64/usr/lib/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
> 482357 brk(NULL) = 0x0000005500042000
> 482357 openat(AT_FDCWD,"sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13",O_RDONLY|O_CLOEXEC) = 3
> 482357 read(3,0x2841280,832) = 832
> 482357 getcwd(0x5500041980,128) = 18
> 482357 mmap(0x0000000000400000,937984,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0)Segmentation fault (core dumped)
The crash is because the guest g++ executable overlaps the host qemu executable, and this
> $ qemu-aarch64-static -strace sid-aarch64/usr/lib/ld-linux-aarch64.so.1 --verify sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13
> 482357 brk(NULL) = 0x0000005500042000
> 482357 openat(AT_FDCWD,"sid-aarch64/usr/bin/aarch64-linux-gnu-g++-13",O_RDONLY|O_CLOEXEC) = 3
> 482357 read(3,0x2841280,832) = 832
> 482357 getcwd(0x5500041980,128) = 18
> 482357 mmap(0x0000000000400000,937984,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0)Segmentation fault (core dumped)
mmap clobbers qemu itself. There is some code in qemu that *attempts* to avoid this
situation, but it *only* works when the main executable is the one with the conflict.
In this case, you're running the g++ executable indirectly, with ld-linux-aarch64.so.1 as
the main executable, so qemu cannot see the conflict.
Solving this is only possible with complete separation of host and guest address spaces.
QEMU does this by default when running 32-bit guest on 64-bit host, but we have no
general-purpose solution for 64-bit guest.
You can avoid some of the problems with QEMU_RESERVED_VA=0x1000000000, giving the guest a
dedicated 40-bit address space, but that will cause other failures with applications that
expect more virtual address space.
It is a long-term goal for QEMU to be able to provide a complete separate virtual address
space, but that will not happen soon.
0 new messages