What is the correct way to use dlopen()?

[Andreas Falkenhahn]

I have a shared object bundled with my Android app. It is in the "lib" folder of the
APK. On Android 7 I can just do the following to load it:

lib = dlopen("libfoo.so", RTLD_LAZY);

However, this doesn't work on Android 4. On Android 4 I have to do the following instead:

lib = dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);

But opening from /data/data/.../ doesn't work on Android 7. So the solution I've come
up with is something like this:

if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib = dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}

This will work on both, Android 4 and Android 7, but of course it feels hackish.
So what is the official way to do it please?

NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.

--
Best regards,
Andreas Falkenhahn mailto:and...@falkenhahn.com

[Andreas Falkenhahn]

Helloo-oo? How come nobody knows this? Isn't this one of the most common things you do with the Android NDK? There surely must be someone here who can answer this!?

[Dan Albert]

NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.

You don't really want call dlclose anyway. There are bugs with thread locals that will cause crashes until P if you dlclose a library: https://github.com/android-ndk/ndk/issues/360

Or leaks prior to r17: https://github.com/android-ndk/ndk/issues/620

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

[J Decker]

https://github.com/d3x0r/SACK/blob/master/src/deadstart/android/client_android_main.c#L462

When I start, I scan /proc/self/maps and find the map which covers the address of the routine I'm in; this gives me the full path of the current code.  I can then strip off the last part of the name and keep the path part to build dlopen paths.

You previously had to manually load shared object libraries that others also depend on as it wouldn't auto-resolve the paths... (libC depends on libB which depends on libA; you have to load A, B and then C manually; loading C won't also bring in A and B... )

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "android-ndk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFVaGhvMD3M%3Dw2BKORqVx%2Bkhs0%2BfWFVTAaij-6qmBAYy9gk4EQ%40mail.gmail.com.

[Andreas Falkenhahn]

I'm not using thread locals or C++ so AFAICS I'm not affected by these issues or is there anything else I should be aware of? My code is really just basic C with pthreads but no thread locals.

[Kenneth Geisshirt]

As I recall, there are some changes in what you can do with dlopen:

https://developer.android.com/about/versions/nougat/android-7.0-changes#ndk

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

--

Kenneth Geisshirt, M.Sc., Ph.D.
Majbøl Allé 18, DK-2770 Kastrup, +45 60 62 71 82

[J Decker]

On Fri, Apr 27, 2018 at 6:41 AM, Kenneth Geisshirt <ken...@geisshirt.dk> wrote:

As I recall, there are some changes in what you can do with dlopen:

https://developer.android.com/about/versions/nougat/android-7.0-changes#ndk

the parts of that - you can't load the system versions; only the local version; but the OP seems to want to load their own version; so none of that really matters.

 

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

--

Kenneth Geisshirt, M.Sc., Ph.D.
Majbøl Allé 18, DK-2770 Kastrup, +45 60 62 71 82

--

You received this message because you are subscribed to the Google Groups "android-ndk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFLzvupxrR2p9V%3DiLqK3zqGzFNq6bPaGp--R1%3DBdAsb2AZq38w%40mail.gmail.com.

[Andreas Falkenhahn]

Yes. I'm still completely puzzled by the fact that it is apparently so very difficult to dlopen() a shared object stored inside the APK. This is such a basic functionality and am I really expected to scan /proc/self/maps and then apply lots of kludges just to be able to dlopen() a shared object stored inside my APK? Seriously Google?

On 27.04.2018 at 22:03 J Decker wrote:

> the parts of that - you can't load the system versions; only the
> local version; but the OP seems to want to load their own version; so none of that really matters.

[J Decker]

On Fri, Apr 27, 2018 at 2:38 PM, Andreas Falkenhahn <and...@falkenhahn.com> wrote:

Yes. I'm still completely puzzled by the fact that it is apparently so very difficult to dlopen() a shared object stored inside the APK. This is such a basic functionality and am I really expected to scan /proc/self/maps and then apply lots of kludges just to be able to dlopen() a shared object stored inside my APK? Seriously Google?

I entirely concur; The LD_LIBRARY_PATH is read at program startup so if you modify that, it doesn't apply.  If you do a dlopen() and there's a dependancy missing, you can't then parse the error and load the missing dependancy and then re-call dlopen() on the first one, because the library caches that it was an error and returns with the same error, even though the symbol/library can now be resolved.

 

On 27.04.2018 at 22:03 J Decker wrote:

> the parts of that - you can't load the system versions; only the
> local version; but the OP seems to want to load their own version; so none of that really matters.

--
Best regards,
 Andreas Falkenhahn                            mailto:and...@falkenhahn.com

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/1310529704.20180427233858%40falkenhahn.com.

[Andreas Falkenhahn]

But maybe the issue has been resolved already because, as I said, starting with Android 7 it is possible to just pass the name of the shared object and it will work, i.e.

lib = dlopen("libfoo.so", RTLD_LAZY);

So maybe the kludge is only needed for older Android versions. Unfortunately, I have no Android 5 and Android 6 test devices here so I don't know about Android 5 and 6. All I can say is that on Android 4 just passing "libfoo.so" definitely doesn't work. It needs a full path on Android 4.

It would really be nice if there was an official word on this. As I said, it looks like it has been resolved already because on Android 7 and up just passing "libfoo.so" works fine.

Maybe I should file a ticket so that this is forwarded to the people who know more about it?

On 28.04.2018 at 04:30 J Decker wrote:

> I entirely concur; The LD_LIBRARY_PATH is read at program startup
> so if you modify that, it doesn't apply. If you do a dlopen() and
> there's a dependancy missing, you can't then parse the error and
> load the missing dependancy and then re-call dlopen() on the first
> one, because the library caches that it was an error and returns
> with the same error, even though the symbol/library can now be resolved.

[Dan Albert]

+dimitry and enh, who hopefully remember the changes and when they happened better than I do...

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/675660551.20180428122739%40falkenhahn.com.

[rpri...@google.com]

On Saturday, April 28, 2018 at 3:27:42 AM UTC-7, Andreas Falkenhahn wrote:

But maybe the issue has been resolved already because, as I said, starting with Android 7 it is possible to just pass the name of the shared object and it will work, i.e.

    lib = dlopen("libfoo.so", RTLD_LAZY);

So maybe the kludge is only needed for older Android versions. Unfortunately, I have no Android 5 and Android 6 test devices here so I don't know about Android 5 and 6. All I can say is that on Android 4 just passing "libfoo.so" definitely doesn't work. It needs a full path on Android 4.

It would really be nice if there was an official word on this. As I said, it looks like it has been resolved already because on Android 7 and up just passing "libfoo.so" works fine.

Maybe I should file a ticket so that this is forwarded to the people who know more about it?

I suspect this is the same issue as https://issuetracker.google.com/36950617, which was fixed for JB-MR2 / API 18 / Android 4.3. I think the fix for that issue added the app's shared library directory to the search path the dynamic loader uses both for dlopen and for finding DT_NEEDED dependencies.

Fixes:

 - https://android-review.googlesource.com/c/platform/libcore/+/48407

 - https://android-review.googlesource.com/c/platform/dalvik/+/48409

 - https://android-review.googlesource.com/c/platform/bionic/+/48534

FWIW: The last time I was interested in testing a bunch of Android versions, I used Android Studio to download and run the x86 AVD emulator images.

On 28.04.2018 at 04:30 J Decker wrote:

> I entirely concur; The LD_LIBRARY_PATH is read at program startup
> so if you modify that, it doesn't apply.  If you do a dlopen() and
> there's a dependancy missing, you can't then parse the error and
> load the missing dependancy and then re-call dlopen() on the first
> one, because the library caches that it was an error and returns
> with the same error, even though the symbol/library can now be resolved.

I think caching the error might have also been fixed, maybe by the Bionic change I linked above. Also see https://issuetracker.google.com/36935779.

-Ryan

[bher...@reification.io]

If you are packaging the .so directly in libs/ that may be your problem.

I'm doing exactly this loading pattern with a shared library and it works without issue if you put it in the correct abi sub-directory

e.g. libs/arm64-v8a/libYouLibrary.so