How to load .so file which depends on other .so file with versiob no.

[vikas44]

I took bambuser ffmpeg, and ran build.sh then NDK build for my
application.
I coppied all generated .so file to libs/armeabi folder to android
java side.
When I trying to load lybrary(System.loadLibrary("avcodec")), its
unable to open the the library. from the logcat I could see "Unable to
dlopen: cannot load library".
With some debug & strace with emulator, I came to know my .so file is
depends on another .so file which has some version number.
Please suggest how to handle this situation? How could remove
dependency from other .so file? or How could I add dependent library
to device.

Dependency looks like that:
vikas@vikas-VirtualBox:~/PlayerC/libs/armeabi$ objdump -x
libavcodec.so | grep NEEDED
NEEDED libavcore.so.0
NEEDED libavutil.so.50
NEEDED libm.so
NEEDED libz.so
NEEDED libc.so

Again those are linked to other shared lib
libavcore.so -> libavcore.so.0.16.0
libavutil.so.50 -> libavutil.so.50.34.0


Please suggest & already I have invested couple of days on that.

Thanks & BR

[Wil Hadden]

Are you trying to build using the ndk build environment, ndk_build or
using makefiles?

A ndk_build build should set the so_name of any libs it builds so that
there is no version number in the name. I have briefly looked at the
bambuser and I did not see a Android.mk file so I presume that it's
not a ndk_build.

So either you
1) Convert the package to use ndk_build, it's not hard it's just a
different type of makefile
2) Roll up your sleeves and try to get the package working, I've done
that on different projects and documented it http://bit.ly/oI8R3u I
would not recommend it though. Pro tip don't use cyginw, use Mingw
shell.
3) Use a different package http://odroid.foros-phpbb.com/t338-ffmpeg-compiled-with-android-ndk
http://stackoverflow.com/questions/5204251/compile-ffmpeg-with-android-ndk-r5b
or hunt around kernel.org if it's up yet.

Wil

[David Turner]

In general, versioned system libraries are not supported by the Android Package Manager (on the device) and the Packaging build tools (on the host).

There are several things you could try:

1/ Try generating libraries without version numbers (e.g. use the -avoid-version option in libtool, instead of -version-info <major>:<minor>), this may require modifying config/build scripts, or simply use the NDK build system to generate the libraries.

2/ Manually remove the version numbers in the generated shared libraries (e.g. replace the first do _after_ "so" by a 0 byte in the symbol table) + rename the libraries

3/ Embed the libraries as assets, manually extract them on first run into the app's data directory, then use System.loadLibrary("<full-path-to-libfoo.so.3.2>"), because System.loadLibrary("foo") will only try to load something called "libfoo.so".

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.

[Martin Storsjö]

Hi,

If you used the bambuser ffmpeg source package, you need to apply the
included patches, too (which is done by extract.sh, as explained in the
README file). Patch #1, "Set the soname to be $(SLIBNAME) instead of
$(SLIBNAME_WITH_MAJOR)", takes care of exactly this issue, by making the
libraries depend on each other via sonames without version number.

// Martin

[Wil Hadden]

On Oct 17, 11:43 am, Martin Storsjö <mar...@martin.st> wrote:

> If you used the bambuser ffmpeg source package, you need to apply the
> included patches, too (which is done by extract.sh, as explained in the
> README file). Patch #1, "Set the soname to be $(SLIBNAME) instead of
> $(SLIBNAME_WITH_MAJOR)", takes care of exactly this issue, by making the
> libraries depend on each other via sonames without version number.
>
> // Martin

Will he not run in to the problem where if he loads, say libA.so which
depends on libB.so, that the kernel will not have has current path in
the so lib path, meaning it will not find libB.so, not unless libB.so
is in a system path?

Wil

[Martin Storsjö]

Hi Wil,

On Mon, 17 Oct 2011, Wil Hadden wrote:

Yes, the patch I mentioned doesn't work around that issue (which is
present for all cases where one .so depends on another .so in the same
apk), it only changes the soname that the library exposes (making
libraries depend on "libavutil.so" instead of "libavutil.so.50".

The issue with transitive dependencies is easy to resolve by simply
loading them in reverse order, e.g.
System.loadLibrary("avutil");
System.loadLibrary("avcore");
System.loadLibrary("avcodec");
System.loadLibrary("yourlib");

This issue is also explained in the android NDK docs (in reference to a
similar case) in docs/CPLUSPLUS-SUPPORT.html.

This way of loading libs with transitive dependencies works since Android
1.6.

For 1.5, the bambuser ffmpeg package contains another workaround, patch
#2, which adds the possibility to add a prefix to the sonames. The build
script in that ffmpeg source package adds the prefix
/data/data/com.bambuser.broadcaster/lib/ to the sonames, so that the
linker explicitly looks for that path regardless of library search paths.
This obviously only works as long as the app is installed into this path
(which afaik is the only one for devices running 1.5), but doesn't seem to
harm devices running newer versions (which are handled by loading them in
reverse order as described above).

// Martin

[Wil Hadden]

On Oct 17, 5:43 pm, Martin Storsjö <mar...@martin.st> wrote:
> Hi Wil,

>

> The issue with transitive dependencies is easy to resolve by simply
> loading them in reverse order, e.g.
> System.loadLibrary("avutil");
> System.loadLibrary("avcore");
> System.loadLibrary("avcodec");
> System.loadLibrary("yourlib");
>
> This issue is also explained in the android NDK docs (in reference to a
> similar case) in docs/CPLUSPLUS-SUPPORT.html.
>
> This way of loading libs with transitive dependencies works since Android
> 1.6.
>
> For 1.5, the bambuser ffmpeg package contains another workaround, patch
> #2, which adds the possibility to add a prefix to the sonames. The build
> script in that ffmpeg source package adds the prefix
> /data/data/com.bambuser.broadcaster/lib/ to the sonames, so that the
> linker explicitly looks for that path regardless of library search paths.
> This obviously only works as long as the app is installed into this path
> (which afaik is the only one for devices running 1.5), but doesn't seem to
> harm devices running newer versions (which are handled by loading them in
> reverse order as described above).
>
> // Martin

Cheers Martin,

I had read about using the full path but for the reason you explain I
avoided it as it could easily blow up in the future.

I have also the reverse load order trick which is a lot nicer, though
it does add a sort of link dependency in your code, which is still
nicer than the alternative. Is it not possible to still have a
circular dependency that this would not solve?

Sorry for all the questions, but I went through this recently and
it's nice to know how to work around it!

The biggest issue I foresaw with the reverse order load though, and
correct me if I'm wrong as I never tried this out, but does this not
mean that every lib you load would need a JNI_OnLoad at a bare
minimum, whereas if you could somehow load the necessary libs as a
dependency then there would not be a need for individual JNI_OnLoads?

Wil

[Martin Storsjö]

On Tue, 18 Oct 2011, Wil Hadden wrote:

As far as I know, it shouldn't blow up in the future either - as I said,
we're using it in production (together with the reverse loading order) on
both 1.5 and a lot of newer devices.

Namely, when using the reverse loading order, it first loads the library
(by looking through the paths for application libraries, finding it in the
app directory and telling the system to load exactly that file). When that
library is loaded (e.g. libavutil.so), and loading the next library
(libavcodec.so), libavcodec.so will have a dependency on
/data/data/app.identifier/lib/libavutil.so. On 1.6 and newer, the dynamic
linker will notice that there's already a "libavutil.so" loaded into the
process, which satisfies the dependency. On 1.5, it doesn't realize
there's already a library loaded with the same name, but it tries to look
for a library with the exact name specified,
/data/data/app.identifier/lib/libavutil.so, which hopefully is found
(assuming all 1.5 devices install the apps into that path).

Therefore, as far as I know, this setup works for all devices, both 1.5
and later.

> I have also the reverse load order trick which is a lot nicer, though
> it does add a sort of link dependency in your code, which is still
> nicer than the alternative. Is it not possible to still have a
> circular dependency that this would not solve?

Libraries should never have circular dependencies - they would be kind of
hard to produce, too, and I'm not sure how a normal dynamic linker would
handle that either. Do you really have such a concrete case?

> Sorry for all the questions, but I went through this recently and
> it's nice to know how to work around it!
>
> The biggest issue I foresaw with the reverse order load though, and
> correct me if I'm wrong as I never tried this out, but does this not
> mean that every lib you load would need a JNI_OnLoad at a bare
> minimum, whereas if you could somehow load the necessary libs as a
> dependency then there would not be a need for individual JNI_OnLoads?

No, not at all. You don't need a JNI_OnLoad function. If it isn't present,
there will be a informative message about it not being found, but
System.loadLibrary() still loads the library just fine.

As I said, the binaries produced by the bambuser ffmpeg build script are
the exact binaries used in production there, without any JNI_OnLoad
functions added or anything. And we're not aware of any issue related to
loading of these libraries.

// Martin

[Wil Hadden]

On Oct 18, 10:35 am, Martin Storsjö <mar...@martin.st> wrote:
>
>
>
> Libraries should never have circular dependencies - they would be kind of
> hard to produce, too, and I'm not sure how a normal dynamic linker would
> handle that either. Do you really have such a concrete case?

No I've not got a real case of this, just wondering if it was at all
possible!

>
> > Sorry for  all the questions, but I went through this recently and
> > it's nice to know how to work around it!
>
> > The biggest issue I foresaw with the reverse order load though, and
> > correct me if I'm wrong as I never tried this out, but does this not
> > mean that every lib you load would need a JNI_OnLoad at a bare
> > minimum, whereas if you could somehow load the necessary libs as a
> > dependency then there would not be a need for individual JNI_OnLoads?
>
> No, not at all. You don't need a JNI_OnLoad function. If it isn't present,
> there will be a informative message about it not being found, but
> System.loadLibrary() still loads the library just fine.
>
> As I said, the binaries produced by the bambuser ffmpeg build script are
> the exact binaries used in production there, without any JNI_OnLoad
> functions added or anything. And we're not aware of any issue related to
> loading of these libraries.
>
> // Martin

That's very interesting, I has presumed that JNI_OnLoad was a pre-
requisite, good to know otherwise.

Thanks again for a very full answer.
Wil

[vikas44]

Thanks Wil, Martin, David for your valuable suggestion.

Due to time constrain, I tried martin's steps. I was able to remove
the dependencies and I tried to open the library in reverse order, its
fine so far. Thanks.
Basically I got the answer for my initial answer, but again facing
problem with generated library.
like If I am calling any function from those library e.g
av_open_input_file from avformat, now control is not coming back.
nothing is happening. I am wondering library is corrupted or some
other issue.

I am looking into that problem. If you have any suggestion or earlier
exp with this, All suggestions are most welcome.


Thank you all once again.
BR.