Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

dlopen always fails with android:debuggable="false"

201 views
Skip to first unread message

Lanre Olokoba

unread,
Dec 17, 2024, 5:02:29 PM12/17/24
to android-ndk
I'm facing an incredibly weird issue. I am building an Android app where I embed shared libraries into the APK, and try to load them into the process at runtime using `dlopen`. Some observations:
  • Testing on Android 14 (Galaxy S21), NDK 26.3.11579264 and 26.3.11579264.
  • I have confirmed that the output APK always contains my library in the ABI-specific `libs` folder:
    • I renamed the shared library file, so it differs from the SONAME of the library. I always call `dlopen("filename.so").
  • As I mentioned above, I've been able to isolate calls to `dlopen` failing to setting `android:debuggable="false"` when building my app. When this happens, here is the output of `dlerror`:
```
dlopen failed: library "24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so" not found
```
  • I assumed that this might have something to do with apps having to declare their native libraries with `<uses-native-library>`, but:
    • Declaring the library in the `AndroidManifest.xml` fails with the same error above.
    • Compiling with target SDK version 30 also fails, same error.
  • I can't enable logging in the linker because...`android:debuggable="false"` 
This is an incredibly, incredibly weird so I'll need pointers from the team.

enh

unread,
Dec 17, 2024, 5:06:11 PM12/17/24
to andro...@googlegroups.com
This is an incredibly, incredibly weird so I'll need pointers from the team.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/android-ndk/f3b1db5a-2827-40ea-acfb-dad4d32e5e44n%40googlegroups.com.

Lanre Olokoba

unread,
Dec 17, 2024, 5:39:33 PM12/17/24
to android-ndk
Oops you caught the wrong link. The one you posted is correct. And yes, my issue seems to be a bug in Android, at least as far as I can tell. The call to `dlopen` should always succeed, as the shared library is bundled in the APK. I tried toggling the `android:extractNativeLibs` flag in the `AndroidManifest.xml` to see if that had any effect--it didn't.

Lanre Olokoba

unread,
Dec 17, 2024, 5:39:39 PM12/17/24
to android-ndk
Oh, and here are the verbose logs of the successful call to `dlopen` when `android:debuggable="true"`:
```
D linker  : dlopen(name="24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so", flags=0x101, extinfo=(null), caller="/data/app/~~FdDnSedzoZua5D5y5iBtyQ==/app.videokit.videokit-TmqbFD9DxVQGhnawfb8jhQ==/lib/arm64/libFunction.so", caller_ns=clns-4@0x7535099e70, targetSdkVersion=32) ...
D linker  : find_library_internal(ns=clns-4@0x7535099e70): task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so, is_dt_needed=0
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so, flags=0x101, search_linked_namespaces=1): calling open_library
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so, flags=0x101, realpath=/data/app/~~FdDnSedzoZua5D5y5iBtyQ==/app.videokit.videokit-TmqbFD9DxVQGhnawfb8jhQ==/lib/arm64/24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so, search_linked_namespaces=1)
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so): Adding DT_NEEDED task: liblog.so
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so): Adding DT_NEEDED task: libFunction.so
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so): Adding DT_NEEDED task: libm.so
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so): Adding DT_NEEDED task: libdl.so
D linker  : load_library(ns=clns-4, task=24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so): Adding DT_NEEDED task: libc.so
D linker  : find_library_internal(ns=clns-4@0x7535099e70): task=liblog.so, is_dt_needed=1
D linker  : find_library_internal(ns=clns-4, task=liblog.so): Already loaded (by soname): /system/lib64/liblog.so
D linker  : find_library_internal(ns=clns-4@0x7535099e70): task=libFunction.so, is_dt_needed=1
D linker  : find_library_internal(ns=clns-4, task=libFunction.so): Already loaded (by soname): /data/app/~~FdDnSedzoZua5D5y5iBtyQ==/app.videokit.videokit-TmqbFD9DxVQGhnawfb8jhQ==/lib/arm64/libFunction.so
D linker  : find_library_internal(ns=clns-4@0x7535099e70): task=libm.so, is_dt_needed=1
D linker  : find_library_internal(ns=clns-4, task=libm.so): Already loaded (by soname): /apex/com.android.runtime/lib64/bionic/libm.so
D linker  : find_library_internal(ns=clns-4@0x7535099e70): task=libdl.so, is_dt_needed=1
D linker  : find_library_internal(ns=clns-4, task=libdl.so): Already loaded (by soname): /apex/com.android.runtime/lib64/bionic/libdl.so
D linker  : find_library_internal(ns=clns-4@0x7535099e70): task=libc.so, is_dt_needed=1
D linker  : find_library_internal(ns=clns-4, task=libc.so): Already loaded (by soname): /apex/com.android.runtime/lib64/bionic/libc.so
```

On Tuesday, December 17, 2024 at 5:06:11 PM UTC-5 enh wrote:

J Decker

unread,
Dec 17, 2024, 8:55:39 PM12/17/24
to andro...@googlegroups.com
you could read the file '/proc/self/maps' and see if the libraries you expect to be there are there... it's possible that different libraries get loaded before your library when debug is enabled, that aren't loaded when debug is disabled.
If your library requires another library in the lib folder, dlopen has previously not wanted to load those, and I had to manually load the dependencies first.   And unfortunately while it is possible to get the error about the required library, if it's loaded after the fact, and the original library attempted again, the library continues to return its original error message.

Is this a new project?  Or are you just updating something existing?

--

enh

unread,
Dec 18, 2024, 10:01:40 AM12/18/24
to andro...@googlegroups.com
On Tue, Dec 17, 2024 at 8:55 PM J Decker <d3c...@gmail.com> wrote:
you could read the file '/proc/self/maps' and see if the libraries you expect to be there are there... it's possible that different libraries get loaded before your library when debug is enabled, that aren't loaded when debug is disabled.
If your library requires another library in the lib folder, dlopen has previously not wanted to load those, and I had to manually load the dependencies first.   And unfortunately while it is possible to get the error about the required library, if it's loaded after the fact, and the original library attempted again, the library continues to return its original error message.

i fixed that back in _jellybean_ (api 18) in 2013 ... i doubt anyone's running that on a galaxy s21!

 
Is this a new project?  Or are you just updating something existing?

On Tue, Dec 17, 2024 at 2:02 PM Lanre Olokoba <olokob...@gmail.com> wrote:
I'm facing an incredibly weird issue. I am building an Android app where I embed shared libraries into the APK, and try to load them into the process at runtime using `dlopen`. Some observations:
  • Testing on Android 14 (Galaxy S21), NDK 26.3.11579264 and 26.3.11579264.
  • I have confirmed that the output APK always contains my library in the ABI-specific `libs` folder:
    • I renamed the shared library file, so it differs from the SONAME of the library. I always call `dlopen("filename.so").
  • As I mentioned above, I've been able to isolate calls to `dlopen` failing to setting `android:debuggable="false"` when building my app. When this happens, here is the output of `dlerror`:
```
dlopen failed: library "24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so" not found
```
  • I assumed that this might have something to do with apps having to declare their native libraries with `<uses-native-library>`, but:
    • Declaring the library in the `AndroidManifest.xml` fails with the same error above.
    • Compiling with target SDK version 30 also fails, same error.
  • I can't enable logging in the linker because...`android:debuggable="false"` 
This is an incredibly, incredibly weird so I'll need pointers from the team.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/android-ndk/f3b1db5a-2827-40ea-acfb-dad4d32e5e44n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.

enh

unread,
Dec 18, 2024, 10:02:10 AM12/18/24
to andro...@googlegroups.com
On Tue, Dec 17, 2024 at 5:39 PM Lanre Olokoba <olokob...@gmail.com> wrote:
Oh, and here are the verbose logs of the successful call to `dlopen` when `android:debuggable="true"`:

and what's in the log when debuggable is false?
 

Lanre Olokoba

unread,
Dec 18, 2024, 10:12:59 AM12/18/24
to android-ndk
Looking at the linker source code, there isn't any obvious branching that would cause this to occur when debuggable is set to false. From tracing control flow, my guess is that the call to `open_library_on_paths` in the `open_library` function is what fails. If this hypothesis is correct, then `ns->get_default_library_paths()` differs depending on the value of `android:debuggable`.

Is there any documentation about the internal differences caused by `android:debuggable`, especially as it relates to the linker or linker configuration for the whole application?

Lanre Olokoba

unread,
Dec 18, 2024, 10:13:02 AM12/18/24
to android-ndk
When `debuggable` is false, I can't enable verbose logging in the linker. As such, all I have is the output of `dlerror` which is:
```
dlopen failed: library "24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so" not found
```

Lanre Olokoba

unread,
Dec 19, 2024, 10:01:58 AM12/19/24
to android-ndk
Doing some more digging and it seems that this issue is simpler. For added context, I am calling `dlopen` from a shared library in my app. Consider the following code:
```
Dl_info dlInfo {};
dladdr(reinterpret_cast<void*>(&SomeLocalFunction), &dlInfo);
std::filesystem::path currentPath = dlInfo.dli_fname;
std::filesystem::path targetPath = currentPath.parent_path() / targetName;
LOGD("Current library path: {}", currentPath.string());
LOGD("Target library path path: {}", targetPath.string());
LOGD("Current library exists: {}", std::filesystem::exists(currentPath));
LOGD("Target library exists: {}", std::filesystem::exists(targetPath));
```
With `debuggable="true"`:
```
I MyApp: Current library path: /data/app/~~ONNmy_lEylVTKSHa-vqM9Q==/app.videokit.videokit-b0uygX-AJLQ5Oi6WRE5juA==/lib/arm64/libSomeLibrary.so
I MyApp: Target library path: /data/app/~~ONNmy_lEylVTKSHa-vqM9Q==/app.videokit.videokit-b0uygX-AJLQ5Oi6WRE5juA==/lib/arm64/24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so
I MyApp: Current library exists: true
I MyApp: Target library exists: true
```
And with `debuggable="false"`:
```
I MyApp: Current library path: /data/app/~~EVUtEznjedbesgCz4CIZwQ==/app.videokit.videokit-dKsMopeiY94tdaaIt54lKw==/lib/arm64/libSomeLibrary.so
I MyApp: Target library path: /data/app/~~EVUtEznjedbesgCz4CIZwQ==/app.videokit.videokit-dKsMopeiY94tdaaIt54lKw==/lib/arm64/24138ad4b81d96ebb12fcbaf9bf5d2610993506ed7689d12fd4f64c74b640a49.so
I MyApp: Current library exists: true
I MyApp: Target library exists: false
```
Some notes:

  1. I suspected that this might have to do with native library extraction logic, but explicitly setting `extractNativeLibs="true"` in the manifest doesn't fix this bug.
  2. The linker explicitly handles dynamic linking from the APK file, but I can't tell if the APK library root path is included in `ns->get_default_library_paths()`; and at runtime I'm not sure how to get the APK path to try to call `dlopen` with an explicit file path to the library in the APK (using the `!/` separator).
  3. As mentioned earlier, `libSomeLibrary.so` and the target library I'm trying to load are side-by-side in the ABI library folder in the APK. I have confirmed this both by unzipping the built APK; and by copying the installed APK back to my computer using `adb` and confirming.
This feels like a minor but incredibly critical oversight.

Lanre Olokoba

unread,
Dec 19, 2024, 5:14:09 PM12/19/24
to android-ndk
More curious findings:

Trying to force the linker into the `open_library_in_zipfile` path also fails with the same "not found" error. In code, I am getting the current app APK path (this is randomized on each app start, so my code handles this) then appending the path to `libSomeLibrary.so` (i.e. the current executing library, not the target library I've been trying and failing to load):

```
std::string dsoPath =
        apkPath.string() +
        "!/lib/arm64-v8a/libSomeLibrary.so";
```

It fails:
```

I MyApp: Custom load hnd: 0x0 err: dlopen failed: library "/data/app/~~VmvIk2_XHMBwf1KdpnKAgg==/app.videokit.videokit-ANvF2125nH5TjSnqIJnkcA==/base.apk!/lib/arm64-v8a/libSomeLibrary.so" not found

```

I also made an observation that might or might not be relevant: when the APK is bundled, native libraries for the `arm64-v8a` ABI are bundled in `lib/arm64-v8a`. But at runtime, when you get the path of the current library using `dladdr`, the relative path is `lib/arm64`. I wonder why.

Lanre Olokoba

unread,
Dec 19, 2024, 5:37:43 PM12/19/24
to android-ndk
I think I found the cause. I reproduced my initial setup, and once the call to `dlopen` failed, checked the contents of `lib/arm64` using the `ls` command in `adb shell`. The library was missing, which sort of vindicates the linker.

Adding a `lib` prefix before the name of the library fixes the issue. I confirmed that once I added the `lib` prefix, the library was present in the `lib/arm64` folder as expected, and the call to `dlopen` succeeded.

In effect, when `android:debuggable="false"`, and your library does not have a `lib` prefix, any calls to `dlopen` will fail. This behaviour has me dumbfounded, but perhaps someone thought it was a good idea. And AFAIK this behaviour isn't documented anywhere.

J Decker

unread,
Dec 20, 2024, 3:51:50 AM12/20/24
to andro...@googlegroups.com
On Thu, Dec 19, 2024 at 2:38 PM Lanre Olokoba <olokob...@gmail.com> wrote:
I think I found the cause. I reproduced my initial setup, and once the call to `dlopen` failed, checked the contents of `lib/arm64` using the `ls` command in `adb shell`. The library was missing, which sort of vindicates the linker.

Adding a `lib` prefix before the name of the library fixes the issue. I confirmed that once I added the `lib` prefix, the library was present in the `lib/arm64` folder as expected, and the call to `dlopen` succeeded.
I had forgotten that gotcha - android was very picky about the name requiring lib and .so; if I had 'ExtraCode.plugin' instead, it wouldn't work... but 'libExtraCode.plugin.so' does; that and I think that's the first time you specifically mentioned what it was named?
 

enh

unread,
Dec 20, 2024, 8:51:19 AM12/20/24
to andro...@googlegroups.com
On Wed, Dec 18, 2024 at 10:12 AM Lanre Olokoba <olokob...@gmail.com> wrote:
Looking at the linker source code, there isn't any obvious branching that would cause this to occur when debuggable is set to false.

yeah, that's why i don't believe you :-)

(it's possible samsung altered something? or it's possible that there's some non-obvious reason why it doesn't work. i'll see if i can reproduce on a pixel device ... when i've finished wearing down the battery to cause it to shutdown so i can reboot into fastboot so i can flash an image that isn't so broken i literally can't even reboot it! you'd be surprised how long you can have a phone continuously with the flashlight on when there's basically nothing at all running on it :-( )
 
From tracing control flow, my guess is that the call to `open_library_on_paths` in the `open_library` function is what fails. If this hypothesis is correct, then `ns->get_default_library_paths()` differs depending on the value of `android:debuggable`.

Is there any documentation about the internal differences caused by `android:debuggable`, especially as it relates to the linker or linker configuration for the whole application?

it shouldn't affect either of those things. from libc's perspective the main difference is that the PR_GET_DUMPABLE prctl is 0, which ... oh, that's interesting. the linker logging code explicitly checks that during setup!

  // The most likely scenario app is not debuggable and
  // is running on a user build, in which case logging is disabled.
  if (prctl(PR_GET_DUMPABLE, 0, 0, 0, 0) == 0) {
    return;
  }

okay, where did that come from?

looks like it was in the original commit (https://android-review.googlesource.com/c/platform/bionic/+/247208) with no justification beyond

"""
Note that for security purposes this option is disabled on user build
for non-debuggable apps.
"""

i assume that's just because at the time people were worried about the various LD_ environment variables that changed _behavior_. i'm not sure i can see what the security angle is here beyond "security through obscurity" which is trivially defeated by just running AOSP instead...

...which is probably the only option available to you. actually, just "can i reproduce this on a pixel device?" is probably a good thing to check too.

(thinking about "should i just delete that PR_GET_DUMPABLE check?" the interesting part is that i'd expected this to be a _performance_ thing --- "avoid all these system property lookups on every dlopen()/dlsym() call for production apps on user builds". that's what's making me nervous about removing this, not "security"...)

Lanre Olokoba

unread,
Dec 30, 2024, 6:53:47 PM12/30/24
to android-ndk
This is an interesting gotcha to have, especially given that AFAIK Linux doesn't require it; and given that it actually works when `android:debuggable=true`. If you can, I'd recommend updating the developer-facing documentation to mention this. And the name of the library was mentioned in the output of `dlerror`, which I shared in my original post.

Lanre Olokoba

unread,
Dec 30, 2024, 6:55:00 PM12/30/24
to android-ndk
Thank you all for the help! And Happy New Year 🎊 

enh

unread,
Jan 9, 2025, 5:25:33 PMJan 9
to andro...@googlegroups.com
ah, yeah, well done on getting to the bottom of this ... this is
indeed a long-standing bug: https://issuetracker.google.com/160129591

i've added another comment there, and hopefully we can finally get
this fixed -- i'm pretty sure 2020 wasn't the _first_ time this bit
someone! -- but mentioning this in the docs is definitely a good idea,
since even if this gets fixed in this year's release, it'll stay
broken on old devices.

thoughts on whether
https://android.googlesource.com/platform/bionic/+/main/android-changes-for-ndk-developers.md
or the dlopen() doc comment in dlfcn.h would have been more useful?
which (if any) did you look at? (if neither, i suspect that the former
has the better web search page ranking, so i'll default to that. and
if/when we finally get this fixed, it'll be useful to say something
there anyway!)
> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/6d2122ba-a995-48b2-ab45-e553c7b3f4bcn%40googlegroups.com.

Lanre Olokoba

unread,
Jan 15, 2025, 12:07:41 PMJan 15
to android-ndk
I did look at this link: https://android.googlesource.com/platform/bionic/+/main/android-changes-for-ndk-developers.md . So having a note there would have been helpful. A note in `dlfcn.h` would have helped, but I think it's better to have something in the docs online (i.e. the former link).

enh

unread,
Jan 15, 2025, 3:09:16 PMJan 15
to andro...@googlegroups.com
cool, i've added a note there
(https://android-review.googlesource.com/c/platform/bionic/+/3454400)
and fixed this in the next release. (there's no public URL for the
fix, sadly, because there was a merge conflict internally, and then i
made further changes internally, and now i'm facing the usual
trade-off of "deal with that just so there's a public link, or go and
fix another bug" :-( )

sorry for the pain, but thanks for helping us finally get this annoyance fixed!
> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/2bf582bf-41ab-4b8e-af9f-4de81061e353n%40googlegroups.com.

Lanre Olokoba

unread,
Jan 15, 2025, 10:23:54 PMJan 15
to android-ndk
Perfect, thanks!
Reply all
Reply to author
Forward
0 new messages