[llvm-dev] Exceptions not working when cross compiling for ARM Cortex M4 with clang and precompiled libraries from ARM GNU GCC Toolchain
Kacper Kowalski via llvm-dev
CMake version: 3.21.3
targeted CPU: STM32L432KC, ARM Cortex M4
-mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16
-nodefaultlibs
--sysroot=${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi
-flto
-fdata-sections -ffunction-sections
# For <iostream>, <string>, ...
-isystem "${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/include/c++/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/"
# For <bits/*>, ...
-isystem "${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/include/c++/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/arm-none-eabi/thumb/v7e-m+fp/hard/"
-fexceptions
-g-mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16
-nodefaultlibs
--sysroot=${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi
-flto
-fdata-sections -ffunction-sections
-Wl,--gc-sections
-g
-flto
-fexceptions
# Path to standard libraries: libc, libm, ...
-L"${ARM_GNU_TOOLCHAIN_PATH}/arm-none-eabi/lib/thumb/v7e-m+fp/hard/"
# Path to libgcc
-L"${ARM_GNU_TOOLCHAIN_PATH}/lib/gcc/arm-none-eabi/${ARM_GNU_TOOLCHAIN_GCC_VERSION}/thumb/v7e-m+fp/hard/"
-lc -lm -lnosys -lstdc++ -lgcc")
I use custom linker script and custom startup file to compile the firmware properly. I attach them both.
try
{
throw std::runtime_error{"Some error!"};
} catch (const std::exception&e)
{
printf("Error: %s\r\n", e.what());
}─── Assembly ─────────────
0x080096d6 ? ldr r3, [r4, #0] # r4 holds the address of "got"
0x080096d8 ? mov r1, r6
0x080096da ? mov r0, r4
0x080096dc ? ldr r6, [r3, #16] # Loaded from the address pointed by (r3 + 16)
0x080096de ? add r2, sp, #4
0x080096e0 ? movs r3, #1
0x080096e2 ? blx r6 # This line crashes
[1] from 0x08008d42 in abort
[2] from 0x08009d08 in __gnu_cxx::__verbose_terminate_handler()
[3] from 0x08008cc2 in __cxxabiv1::__terminate(void (*)())
[4] from 0x08008cf2 in std::terminate()
[5] from 0x08009c08 in __cxa_throw
[6] from 0x0800fedc in test_throws_uncaught_exception()+52 at /home/kacper/Workspace/Aura/tests/device/test_serial_logger/test_logs_uncaught_exception.cpp:11
[7] from 0x08014ff0 in run_test(void (*)(), char const*, unsigned int)+92 at test_logs_uncaught_exception_runner.cpp:62
[8] from 0x08014f8e in test_main()+12 at test_logs_uncaught_exception_runner.cpp:79
[9] from 0x08014440 in app_main::$_0::operator()() const+16 at /home/kacper/Workspace/Aura/tests/device/setup/test_runner.cpp:24
Kacper Kowalski via llvm-dev
Kacper Kowalski via llvm-dev
Eli Friedman via llvm-dev
Some small suggestions:
LTO makes everything more complicated; make sure things work without -flto before trying LTO.
A “.got” section shows up when an object file compiled with -fPIC/-fPIE accesses an external global variable or constant. You can check if an object file contains a relocation like this with objdump -r (something like R_ARM_GOT_PREL). For a static link, the linker should fix it to point to the right address. If the linker isn’t computing the correct address, my best guess is that something is wrong with the linker script. Are you setting the address of flash/RAM correctly? Is the difference between the address in the GOT and the actual address related to the size of some section? (The consequences of getting this wrong might not be immediately obvious if the code is mostly using pc-relative addressing.)
If you have a working build with gcc, you might want to compare to see what, exactly, is different. Check the link line with “-###”; are you using the same linker? The same libraries?
-Eli
From: llvm-dev <llvm-dev...@lists.llvm.org> On Behalf Of
Kacper Kowalski via llvm-dev
Sent: Saturday, November 20, 2021 6:22 AM
To: llvm-dev <llvm...@lists.llvm.org>
Subject: [llvm-dev] Exceptions not working when cross compiling for ARM Cortex M4 with clang and precompiled libraries from ARM GNU GCC Toolchain
WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
Peter Smith via llvm-dev
One more small suggestion. Eli’s suggestions are well worth trying first.
If you are static linking the linker should have written the relocated value into the .got section leaving no dynamic relocation. Can you use the linker map file to find that section in the ELF file, presumably it will be in FLASH and check that the resulting value in the flash is correct (assuming the .got will be in the correct place) after copying to RAM. If it is correct in the FLASH but not after copying to RAM then it could possibly be something to do with copying the GOT to the correct address in RAM, i.e. if that has gone wrong then the GOT and various other bits of the program could also be wrong.
I’d also look at the map file to see if other orphan sections (those that haven’t been explicitly placed by your script) have missed out being copied into RAM. To be sure your script hasn’t missed anything I recommend setting --orphan-handling-mode=warn or =error https://sourceware.org/binutils/docs/ld/Options.html#index-_002d_002dorphan_002dhandling_003dMODE
Peter
Kacper Kowalski via llvm-dev
.got :
{
. = ALIGN(8);
*(.got)
*(.got*)
. = ALIGN(8);
} >FLASH
Peter Smith via llvm-dev
IIUC then LLD with libc++, libc++abi, and libunwind works, but no other combination of c++ libraries has worked?
Although I don’t have a source for it, my understanding is that it is not recommended to mix runtime support libraries. For example libc++abi needs to be matched with llvm libunwind and possibly compiler-rt. The same goes for the GNU equivalents.
Splitting things up the problem is likely to be one or more of:
- Code generation of exceptions in clang not being understood by GNU exception handling code
- LLD is unable to link GNU exception handling code correctly
- The combination of runtime libraries that you’ve chosen isn’t working together
One possible experiment to see if it is LLD is to try compiling with clang but linking with the GNU linker. If that starts to work, then can you look to see what the differences are? LLD has a –reproduce option that you can upload to bug-report, although you’ll have to wait for the Bugzilla migration to finish before doing that. If that doesn’t work then it is either some problem/incompatibility with the GNU runtime libraries or an incompatible set of libraries.
There are some things that can be done to reduce the size of libc++ and libc++abi. The demangler in libc++abi is large and only used in error messages so it can be omitted with a few tweaks. Anything that avoids bringing in locale, which is often not needed in embedded systems will also help.
I’m still puzzled at why you are seeing so many GOT entries being created. As Eli mentioned the linker will create a GOT entry in response to specific relocation types like R_ARM_GOT_PREL. The compiler shouldn’t be generating these for a non-PIC and non-PIE build. One possibility is that some later flag on the command line is overriding the -fno-pic, it could be the use of LTO as the code-generator may change the PIC setting from what has been used to make the bitcode objects.
That’s probably the best I can do without seeing the objects.
Peter
Kacper Kowalski via llvm-dev
Peter Smith via llvm-dev
Thanks for the reproducer.
I think I’ve worked out where your GOT entry is coming from. There is a R_ARM_TARGET2 relocation which takes on a different value based on the target. For Linux targets this defaults to R_ARM_GOT_PREL. For embedded systems I think R_ARM_ABS32 is what an embedded build of GNU ld will use.
Can you add -Wl,--target2=abs to your command line (option is supported by GNU ld and LLD; documented https://sourceware.org/binutils/docs/ld/ARM.html). That should make the GOT unnecessary. Ideally we should add that flag to the bare-metal driver for Arm Targets.
There is a chance that this will make everything magically work. If the unwinder is expecting R_ARM_TARGET2 to be resolved as R_ARM_ABS32 then even if lld correctly resolves R_ARM_GOT_PREL it won’t be what the unwinder is expecting.
My apologies I don’t have a lot of spare time today to try and see if I can spot anything else.
Peter
P.S. I’ve removed the message text from before your previous reply as it exceeded the unmoderated message size limit and I don’t expect too many of them to be around today.
From: Kacper Kowalski <kacper.s...@gmail.com>
Sent: 26 November 2021 11:21
To: Peter Smith <Peter...@arm.com>
Cc: Eli Friedman <efri...@quicinc.com>; llvm...@lists.llvm.org
Subject: Re: [llvm-dev] Exceptions not working when cross compiling for ARM Cortex M4 with clang and precompiled libraries from ARM GNU GCC Toolchain
Hello Peter, Eli,
Kacper Kowalski via llvm-dev
Peter Smith via llvm-dev
Thanks for the reproducer.
I think I’ve worked out where your GOT entry is coming from. There is a R_ARM_TARGET2 relocation which takes on a different value based on the target. For Linux targets this defaults to R_ARM_GOT_PREL. For embedded systems I think R_ARM_ABS32 is what an embedded build of GNU ld will use.
Can you add -Wl,--target2=abs to your command line (option is supported by GNU ld and LLD; documented https://sourceware.org/binutils/docs/ld/ARM.html). That should make the GOT unnecessary. Ideally we should add that flag to the bare-metal driver for Arm Targets.
There is a chance that this will make everything magically work. If the unwinder is expecting R_ARM_TARGET2 to be resolved as R_ARM_ABS32 then even if lld correctly resolves R_ARM_GOT_PREL it won’t be what the unwinder is expecting.
My apologies I don’t have a lot of spare time today to try and see if I can spot anything else.
Sent: 26 November 2021 11:21