Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

OpenMP | Stack Canary Support -fstack-protector-strong

329 views
Skip to first unread message

Javier Gerardo Martinez Salomon

unread,
Sep 12, 2024, 9:49:42 AM9/12/24
to android-ndk
Hi all,

I'm working in an Android project that requires to use OpenMP in different modules simultaneously, I've initially linked OpenMP statically, however, that caused issues since OpenMP can't deal with multiple instances of it running and throws an error during initialization(error #15).

In the end I decided to link with OpenMP dynamically and found the following resource in which the NDK dev team discusses adding a dynamic version of OpenMP: 

//In the CMakeLists.txt
target_link_libraries(mylib -fopenmp)
//In the build.gradle
cmake {
arguments "-DANDROID_STL=none"
cppFlags "-fopenmp"
cFlags "-fopenmp"
}

I'm using NDK r25c, and I was able to copy the pre-built `libomp.so` shared libraries to the `jniLibs` of a common module so that the rest of modules could use it once it's packed into an APK, this works as expected, there are no multiple copies of the OpenMP runtime, the code that uses OpenMP gets the boost it needs and I didn't had to use workarounds such as `KMP_DUPLICATE_LIB_OK=TRUE`, however, while conducting a security analysis I found that the `libomp.so` libs within the NDK do not export any __chk symbols that correspond to fortified functions nor stack canary protection functions and I want to ask some questions about it:
  1. Is there a reason why these libs aren't built with the -fstack-protector-strong flag?
  2. Are there plans to distribute the libs with stack canary protection support in newer versions of the NDK?
  3. Do I need to build libomp.so myself to add support for it, and if so, is there any resource I can use to build it myself or add support for stack canary protection in another way?
Any lead is welcome, in the end I want to either document the reason why this lib is missing such protections or mitigate it myself if there are no plans to add support for it in the future.

Thank you in advance

Dan Albert

unread,
Sep 12, 2024, 6:05:58 PM9/12/24
to android-ndk, Pirama Arumuga Nainar, Elliott Hughes
  1. Is there a reason why these libs aren't built with the -fstack-protector-strong flag?
  2. Are there plans to distribute the libs with stack canary protection support in newer versions of the NDK?
They should be already. Both are on by default in the NDK's CMake toolchain file, which is used to build all the toolchain runtime libraries. Unless they've opted out for some reason, but I don't see that: https://cs.android.com/search?q=stack-protector-strong&ss=android-llvm and https://cs.android.com/search?q=FORTIFY_SOURCE&sq=&ss=android-llvm.

The problem is that FORTIFY was implemented gradually in Android, so the runtime support for that isn't necessarily available in your minSdkVersion, and the _chk functions can only be called if your minSdkVersion is new enough for them to be guaranteed. The extra layer of the problem here is that even if your minSdkVersion is 35 and would be able to use all the FORTIFY _chk runtime stuff, the NDK has only one libomp.so, and that's built for the minimum minSdkVersion that NDK allows (idr about r25, that's been out of support for a while, but r27's is 21), so it'll only be able to call the _chk functions that existed back in API 21 (some, but not a lot). I think when you don't have the runtime support, you still get compile-time support (+Elliott Hughes, right?), but you're right that that's a much less effective mitigation.

That doesn't explain why you wouldn't see any references though (I hope you mean references and not exports... I've seen it happen, but that'd be a bug, because libc is the provider for those). My first guess was that they'd be inlined hidden and then the linker would eliminate the references, but they can't be inlined afaik. The only plausible explanation seems to be that my analysis in the first paragraph is wrong and the runtimes are not being built with those flags... +Pirama Arumuga Nainar.

I guess it could also be the case that none of the code in OMP called any of the fortified functions, nor needed stack protectors? That seems unlikely, but I think it is possible. idr when the compiler is able to elide __stack_chk_guard, but I think there are cases where it may.

FWIW it doesn't look like this is any different for the static or shared runtimes.

~/Library/Android/sdk/ndk/27.1.12297006/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-readelf -sW ~/Library/Android/sdk/ndk/27.1.12297006/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/aarch64/libomp.so | grep _chk
~/Library/Android/sdk/ndk/27.1.12297006/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-readelf -sW ~/Library/Android/sdk/ndk/27.1.12297006/toolchains/llvm/prebuilt/darwin-x86_64/lib/clang/18/lib/linux/aarch64/libomp.a | grep _chk

Both find no results.

//In the build.gradle
cmake {
arguments "-DANDROID_STL=none"
cppFlags "-fopenmp"
cFlags "-fopenmp"
}

While it's not the cause of the problem, using cppFlags and cFlags in gradle is not advisable. In some circumstances (I don't remember which, and iirc it's complicated anyway) it will override the default flags required to build correctly for Android. The default stack-protector and FORTIFY flags would be impacted by that  problem if you have run into it, I believe.

idk why the properties were added to the gradle DSL. I imagine it was so that you can control build variant specific-flags from gradle where it's slightly more ergonomic to do so, but the downsides aren't worth it. https://developer.android.com/ndk/guides/cmake#manage_compiler_flags explains this in more detail and suggests some alternatives if that's what you need.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/41f6e565-fb8e-4edf-bcd4-a11f1725b7cdn%40googlegroups.com.

Dan Albert

unread,
Sep 12, 2024, 7:11:15 PM9/12/24
to Pirama Arumuga Nainar, android-ndk, Elliott Hughes

On Thu, Sep 12, 2024 at 3:54 PM Pirama Arumuga Nainar <pir...@google.com> wrote:
The toolchain libraries don't use the NDK's CMake toolchain file.  The flags used to build the runtime libraries such as libc++ and openmp are explicitly configured in the llvm-toolchain branch.  We should be building these with FORTIFY_SOURCE and -fstack-protector.  (The toolchain for r28 is almost finalized so this could be done for r29).

Javier Gerardo Martinez Salomon

unread,
Sep 12, 2024, 8:29:12 PM9/12/24
to android-ndk
Hi Dan,

Thank you for your detailed answer, the security scanning tool I'm using analyzes the binaries with the `lief` python library to read the ELF and determine the presence of fortified functions, I tried myself with objdump but no dice either, e.g:
objdump -d ~/Library/Android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/14.0.7/lib/linux/aarch64/libomp.so | grep _chk

but of course my results match the same as yours and the scanning tool, in the past I had taken the OpenMP runtime static library and my guess is that by analyzing the binary that linked libomp.a instead of libomp.so some of the functions are found(likely pertaining to the consumer lib) and so the scanning tool was happy.

Is there anything you can recommend to get such protection sooner? would I need to rebuild the libomp.so specifying the missing flags? or is it necessary to wait until NDK r29 is release?

Thanks for your help

Dan Albert

unread,
Sep 12, 2024, 8:36:13 PM9/12/24
to andro...@googlegroups.com
Is there anything you can recommend to get such protection sooner? would I need to rebuild the libomp.so specifying the missing flags?

Yeah, if it's urgent you can build libomp.so yourself. It won't be fun: https://android.googlesource.com/toolchain/llvm_android/+/main/BUILD.md

If it's only semi urgent (can't wait 6-9 months for r29, but can wait a few months), once Pirama has the changes landed in AOSP you'd be able to pull those binaries from ci.android.com. I wouldn't normally recommend shipping toolchain components from an unqualified build, but in this case it's not really any different than you building them yourself. I can't promise any timeline on that though. It'd be "whenever the next toolchain update is ready" and that's never been an easy thing to estimate reliably.

Javier Gerardo Martinez Salomon

unread,
Sep 13, 2024, 12:10:47 AM9/13/24
to android-ndk
Thanks for the recommendation, I figured that building libomp.so myself wouldn't be an easy task, and on top of it, it's already problematic to include a dynamic copy of the libomp.so runtime, documenting the NDK version used to build the libraries helps to maintain a record on where the lib is coming from, building my own makes it hard to enforce that.

I'll be looking forward to hear news about the future development, I'm glad I could help you notice the missing flags, thanks.

enh

unread,
Sep 13, 2024, 10:51:23 AM9/13/24
to Dan Albert, George Burgess, android-ndk, Pirama Arumuga Nainar
On Thu, Sep 12, 2024 at 6:05 PM Dan Albert <dana...@google.com> wrote:
  1. Is there a reason why these libs aren't built with the -fstack-protector-strong flag?
  2. Are there plans to distribute the libs with stack canary protection support in newer versions of the NDK?
They should be already. Both are on by default in the NDK's CMake toolchain file, which is used to build all the toolchain runtime libraries. Unless they've opted out for some reason, but I don't see that: https://cs.android.com/search?q=stack-protector-strong&ss=android-llvm and https://cs.android.com/search?q=FORTIFY_SOURCE&sq=&ss=android-llvm.

The problem is that FORTIFY was implemented gradually in Android, so the runtime support for that isn't necessarily available in your minSdkVersion, and the _chk functions can only be called if your minSdkVersion is new enough for them to be guaranteed. The extra layer of the problem here is that even if your minSdkVersion is 35 and would be able to use all the FORTIFY _chk runtime stuff, the NDK has only one libomp.so, and that's built for the minimum minSdkVersion that NDK allows (idr about r25, that's been out of support for a while, but r27's is 21), so it'll only be able to call the _chk functions that existed back in API 21 (some, but not a lot). I think when you don't have the runtime support, you still get compile-time support (+Elliott Hughes, right?), but you're right that that's a much less effective mitigation.

+George Burgess for the canonical answer, but i don't think so --- the random sample i just checked are all or nothing.

something's a bit weird though, because (copy and pasting from the fortify section of my "prospective polyfills" doc) there are relatively few post-21 fortified functions, and almost all of the string/memory routines are pre-21:

    __memchr_chk; # introduced=23
    __memrchr_chk; # introduced=23
    __poll_chk; # introduced=23
    __ppoll_chk; # introduced=23
    __ppoll64_chk; # introduced=28
    __pread64_chk; # introduced=23
    __pread_chk; # introduced=23
    __readlink_chk; # introduced=23
    __readlinkat_chk; # introduced=23
    __fread_chk; # introduced=24
    __fwrite_chk; # introduced=24
    __getcwd_chk; # introduced=24
    __pwrite_chk; # introduced=24
    __pwrite64_chk; # introduced=24
    __write_chk; # introduced=24
    __mempcpy_chk; # introduced=30
    __sendto_chk; # introduced=26
 
(oh, i see you solve that mystery later in the thread. i'll leave this here though, because it's mildly interesting and not otherwise directly recorded anywhere public.)

enh

unread,
Sep 18, 2024, 11:57:39 AM9/18/24
to George Burgess, Dan Albert, android-ndk, Pirama Arumuga Nainar
On Fri, Sep 13, 2024 at 10:58 AM George Burgess <gb...@google.com> wrote:
> I think when you don't have the runtime support, you still get compile-time support (+Elliott Hughes, right?), but you're right that that's a much less effective mitigation

It's spotty. As time's gone on, some of the FORTIFY compile-time checks have moved into Clang, so everyone with `-Wfortify-source` gets them.

For the Bionic-specific compile-time checks, it's currently all-or-nothing like enh@ says. You need to enable _FORTIFY_SOURCE to get the compile-time checks, and _FORTIFY_SOURCE implies run-time checks unless you're using ASAN.

If we want a middle-ground, it's probably better to push more warnings into Clang's `-Wfortify-source` (benefitting all platforms, and letting us remove complexity from Bionic) than to add yet another configuration to Bionic's FORTIFY.

(i think everyone knows i already feel like this, but just for the record: yes, i think moving ever more of this into clang is by far the best choice. and not just for bionic: it would help me from the perspective of "i'd like to move the AOSP host stuff to musl, but would prefer not to lose fortify" too, for example.)
 
--
Folks have varying work hours; if I caught you outside of yours, please don't feel pressured to respond immediately.
Reply all
Reply to author
Forward
0 new messages