loading library with dependencies

[fnizou]

Hello,

My issue is quite simple, i have two libraries libm1.so and libm2.so.
libm1.so is linked on libm2.so.
System.loadLibrary("m1") fails, the message in adb logcat clearly
states libm2.so was not found.

D/dalvikvm( 179): Trying to load lib /data/data/
com.example.android.helloactivity/lib/libm1.so 0x43757800
I/dalvikvm( 179): Unable to dlopen(/data/data/
com.example.android.helloactivity/lib/libm1.so): Cannot load library:
link_image[1638]: 29 could not load needed library 'libm2.so' for
'libm1.so' (load_library[984]: Library 'libm2.so' not found)
W/System.err( 179): java.lang.UnsatisfiedLinkError: Library m1 not
found
W/System.err( 179): at java.lang.Runtime.loadLibrary(Runtime.java:
489)

If i do System.loadLibrary("m2"); System.loadLibrary("m1"); then it
works.

Why can't the System load itself m2 since it located m1 successfully.
Is is a bug, a limitation, expected? Is there any bypass?
I tried to modify the property "java.library.path" to point also on /
data/data/com.example.android.helloactivity/lib without success.
I also tried to directly use dlopen but the behavior is similar (i
guess LoadLibrary calls dlopen internally, no surprise)

My test are on emulator only from 1.5 to 3.2 with the same results.

By the way, the previous example is just a test but we have a more
complex legacy with many apps where loading library in the proper
order in the java code is not much an option.

Thank you,

François

[David Turner]

On Thu, Nov 17, 2011 at 4:13 PM, fnizou <francoi...@gmail.com> wrote:

Hello,

My issue is quite simple, i have two libraries libm1.so and libm2.so.
libm1.so is linked on libm2.so.
 System.loadLibrary("m1") fails, the message in adb logcat clearly
states libm2.so was not found.

D/dalvikvm(  179): Trying to load lib /data/data/
com.example.android.helloactivity/lib/libm1.so 0x43757800
I/dalvikvm(  179): Unable to dlopen(/data/data/
com.example.android.helloactivity/lib/libm1.so): Cannot load library:
link_image[1638]:    29 could not load needed library 'libm2.so' for
'libm1.so' (load_library[984]: Library 'libm2.so' not found)
W/System.err(  179): java.lang.UnsatisfiedLinkError: Library m1 not
found
W/System.err(  179):    at java.lang.Runtime.loadLibrary(Runtime.java:
489)

If i do System.loadLibrary("m2"); System.loadLibrary("m1"); then it
works.

Yes, and this is the documented behaviour: you must load libraries in reverse dependency order explicitely.

 

Why can't the System load itself m2 since it located m1 successfully.
Is is a bug, a limitation, expected? Is there any bypass?
I tried to modify the property "java.library.path" to point also on /
data/data/com.example.android.helloactivity/lib without success.
I also tried to directly use dlopen but the behavior is similar (i
guess LoadLibrary calls dlopen internally, no surprise)

It is a limitation of the system.

In a nutshell, the dynamic linker doesn't know anything about your application (e.g. where its libraries live), it only knows about the LD_LIBRARY_PATH value that was set when the process was created. When you start an Android application, you really fork the Zygote process, you don't create a new one, so the library search path is the initial one and doesn't include your app's /data/data/<pkgname>/lib/ directory, where your native libraries live. This means that dlopen("libfoo.so") will not work, because only /system/lib/libfoo.so will be searched.

When you call System.loadLibrary("foo") from Java, the VM framework knows the application's directory, so it can translate "foo" into "/data/data/<pkgname>/lib/libfoo.so", then call dlopen() with this full path, which will work.

It libfoo.so references "libbar.so", then the dynamic linker will not be able to find the latter.

Add to this that even if you update LD_LIBRARY_PATH from native code, the dynamic linker will not see the new value. For various low-level reasons, the dynamic linker contains its own copy of the program's environment as it was when the process was created (not forked). And there is simply no way to update it from native code. This is by design, and changing this would have drastic security constraints. For the record, this is also how the Linux dynamic linker works, this forces any program that needs a custom library search path to use a wrapper script to launch its executable (e.g. Firefox, Chrome and many others).

We could hack the dynamic linker and framework to solve this in various secure ways, but at the moment loading the libraries in reverse order doesn't appear as a big bottleneck.

 

My test are on emulator only from 1.5 to 3.2 with the same results.

By the way, the previous example is just a test but we have a more
complex legacy with many apps where loading library in the proper
order in the java code is not much an option.

Sorry, but I don't think there is reliable work-around. Some people have been playing with rpath to embed the full path of dependent libraries in the .so itself, but this is no guaranteed to work (e.g. it will break if you move the app to the SD Card, because now the libraries are in a directory with a completely different path, most of it random iirc).

 

Thank you,

François

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.

[Tor Lillqvist]

See the function lo_dlopen() in http://cgit.freedesktop.org/libreoffice/core/tree/sal/osl/android/jni/lo-bootstrap.c , and the helper functions it calls, and the code at http://cgit.freedesktop.org/libreoffice/core/tree/sal/osl/android/src/org/libreoffice/android/Bootstrap.java#n76 that tells where to look for the app's shared libraries. This code is not really generally re-usable as such (and the license of course might be a problem), but hopefully should give some ideas.

--tml

[Tor Lillqvist]

Tö be more verbose, what that lo_dlopen() function does is:

- Searches where the shared object in question is. It searches a set of directories passed to it by the Java code. The Java code looks at LD_LIBRARY_PATH and adds the app's lib directory to that.
- Opens the found shared object file and reads the ELF structures in it . Not all, but just enough to find out what shared objects it needs (the DT_NEEDED ones as displayed by arm-linux-androideabi-readelf -d). It calls itself recursively on the needed shared objects.
- Only after that, i.e. after making sure that all needed other shared objects have been loaded, it calls the real dlopen() on the found full pathname to the shared object.

A few gotchas that the code handles:

- You should not call dlopen() unless you are sure it will succeed. The Bionic dynamic linker "helpfully" remembers if a dlopen() call has failed and refuses to even try again dlopening the same library a second time, even if you after the first attempt have loaded the required dependencies that caused the failure the first time. I.e. it is not a good idea to just try dlopen() first and if it fails, parse the dlerror() message...

- The wrapper needs to remember what libraries it already has loaded to avoid infinitely looping when libraries have circular dependencies (which they do have).

--tml

[fnizou]

Thank you for the insight. The how and why is more clear now. Now that i know its not a "bug", i will find something else, either a wrapper or maybe generating java code buildtime with the System.loadLibrary in the correct order.
I would not do the rpath trick since we generate each .so one time but they can be packaged in more than one app, not to mention the SD card thing.

François

[fnizou]

I will have a look at lo_dlopen(), it seems an interesting approach. Thank you!

François

[David Turner]

On Fri, Nov 18, 2011 at 3:49 PM, Tor Lillqvist <t...@iki.fi> wrote:

Tö be more verbose, what that lo_dlopen() function does is:

- Searches where the shared object in question is. It searches a set of directories passed to it by the Java code. The Java code looks at LD_LIBRARY_PATH and adds the app's lib directory to that.
- Opens the found shared object file and reads the ELF structures in it . Not all, but just enough to find out what shared objects it needs (the DT_NEEDED ones as displayed by arm-linux-androideabi-readelf -d). It calls itself recursively on the needed shared objects.
- Only after that, i.e. after making sure that all needed other shared objects have been loaded, it calls the real dlopen() on the found full pathname to the shared object.

A few gotchas that the code handles:

- You should not call dlopen() unless you are sure it will succeed. The Bionic dynamic linker "helpfully" remembers if a dlopen() call has failed and refuses to even try again dlopening the same library a second time, even if you after the first attempt have loaded the required dependencies that caused the failure the first time. I.e. it is not a good idea to just try dlopen() first and if it fails, parse the dlerror() message...

Thank you Tor, your code is very insightful. For the record, this is the first time I heard about this linker bug. Is there an entry for it on b.android.com? It sure looks something that could be fixed relatively easily in the platform.

 

- The wrapper needs to remember what libraries it already has loaded to avoid infinitely looping when libraries have circular dependencies (which they do have).

--tml

--

You received this message because you are subscribed to the Google Groups "android-ndk" group.

To view this discussion on the web visit https://groups.google.com/d/msg/android-ndk/-/mjnj5P9UcIkJ.

[Tor Lillqvist]

Filed bug http://code.google.com/p/android/issues/detail?id=22143

--tml