Building dynamic libraries with undefined references.

1,459 views
Skip to first unread message

Raj Shekar

unread,
Aug 1, 2019, 8:15:05 AM8/1/19
to andro...@googlegroups.com

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

J Decker

unread,
Aug 1, 2019, 11:44:48 AM8/1/19
to andro...@googlegroups.com
On Thu, Aug 1, 2019 at 5:15 AM Raj Shekar <rajash...@gmail.com> wrote:

Hi,

    I'm trying to build a dynamic library with some undefined references (basically calling some functions which belong to other dynamic libraries and they will be loaded before loading this library) from Android Studio. But with any or all the flags as below I'm getting an error as 'error: undefined reference to 'test_func''. 'test_func' is a function which will be called from a.so, but can be exposed by either b.so or c.so. I would like the function test_func( ) to be linked to a library which I would load with 'dlopen( )' at runtime (b.so or c.so), so that I can load different libraries which expose test_func( ) method for different invocations of my app. Can any one please tell me the correct flags to be set in my CMakeLists.txt file to achieve this or if cannot be achieved in Android-Ndk. This is similar to what Linux allows, so I guess I'm missing something.
If you're using dlopen to load the library, then you should use dlsym to get the function from that library and just declare a `void (*test_func)(...)` which is a function pointer to set from the dlsym then just call test_func();
then it doesn't need to exist ever.
Otherwise you'll need a dummy library of some name, but even then, when it links to the test_func in THAT library, it would refer to that library's name, and not nesscarily be obscured by loading other libraries also containing that symbol.
 

Tried below flag settings in CMakeLists.txt
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, --allow-shlib-undefined" )
set( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl, -undefined" )

set(CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_CREATE_CXX_FLAGS} -undefined dynamic_lookup")

Android NDK version:
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

Thanks,
Raj.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAOpHf7MqgZEPwbgEPjgdNO5%3DxG7AmbdLc559fipy2CF948dsxA%40mail.gmail.com.

Raj Shekar

unread,
Aug 2, 2019, 5:37:52 AM8/2/19
to andro...@googlegroups.com
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined


Thanks, 
Raj. 

J Decker

unread,
Aug 2, 2019, 8:12:53 AM8/2/19
to andro...@googlegroups.com
On Fri, Aug 2, 2019 at 2:37 AM Raj Shekar <rajash...@gmail.com> wrote:
Thanks for your time J Decker,  Instead of using the function pointers won't it be possible to link the library with the functions which are exposed by already loaded libraries. I would like to give a try by disabling the --no-undefined flag from the linker options. 

With verbose logging of cmake, how can we disable the - - no-undefined flag from the linker flags, it seems to get added by default. 

LANGUAGE_COMPILE_FLAGS = -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -mstackrealign -fno-addrsig -Wa,--noexecstack -Wformat -Werror=format-security -stdlib=libc++ -std=c++11 -frtti -fexceptions -O0 -fno-limit-debug-info
LINK_FLAGS = -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-allow-shlib-undefined -Wl,-undefined

Relocations are rarely done after a library is loaded.
If the library to use the undefined function doesn't previously resolve the function, it won'tload.  If it does, then loading a new one won't replace the old relocation.
A function pointer for most purposes is the same as the name of a function... inporting a function or function pointer in Java looks the same as any other function...

Raj Shekar

unread,
Aug 14, 2019, 2:51:35 AM8/14/19
to andro...@googlegroups.com

 Thanks for your time and inputs J Decker, and due to some unavoidable circumstances I couldn't reply earlier. Now, I'm using the dlopen to load the libraries at run time and link the function pointers using dlsym (instead of relying on undefined references). Got to know that the best way to load the libraries is to use System.loadLibrary("lib.so") calls from Java side. But I want to explicitly do it from C side using dlopen( ) calls only. So, as of now the libraries are placed at '/data/data/<package>/libs' and I'm using this path, but the problem is that the path is undocumented. As per the answer in (https://stackoverflow.com/questions/15719149/how-to-integrate-native-runtime-library-with-dlopen-on-ndk), the best way to obtain the path to the libraries is to use '/proc' and 'derive from that the path to other libs' - I couldn't find more details on how to use '/proc' to obtain the path to the libraries. Can anyone please point me to a tutorial or an example from which I can refer.

  

   Another question: If we have an update to a dynamic library, during my program start-up, can I copy the updated library from another folder to my project libraries folder ('/data/data/<package>/libs' or the one obtained with the help of '/proc'). We would like to do this to avoid giving an updated APK when ever we would have an update to one of our dynamic libraries ? Is it recommended not to use such methods or is it valid on Android platform.. 

 

Thanks,

Raj.


Dan Albert

unread,
Aug 14, 2019, 2:57:13 AM8/14/19
to android-ndk
That path isn't documented because it's not guaranteed. On newer versions of the OS your app is installed to a semi-random location. Anything hard coded will be wrong.

But that doesn't matter. You don't need to give dlopen the full path. Just load the library by name (the file name without the path) and it will be found.

Alex Cohn

unread,
Aug 14, 2019, 4:17:34 AM8/14/19
to android-ndk
The /data/data/<your.package>/lib path is not only undocumented, it's not valid anymore. The path to installed native libs depend on the version of Android, and the most recent versions randomize this path for further protection. The supported way to find location of the native libraries is through Java API ApplicationInfo.nativeLibDir, but even this has some caveats, see htps://stackoverflow.com/a/56551499/192373.

BR,
Alex Cohn

Thanks, 
Raj. 

To unsubscribe from this group and stop receiving emails from it, send an email to andro...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to andro...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to andro...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to andro...@googlegroups.com.

J Decker

unread,
Aug 14, 2019, 6:36:33 AM8/14/19
to andro...@googlegroups.com
Well, I assumed that one library would be with all the other libraries so I have this bit of code


that reads /proc/self/maps and finds its own name (defined as part of the compile options TARGET_NAME as a string), and gets the path from that.  Works on any system that puts libraries in some place. 

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/4ee4455f-7ce3-45a9-bf3c-5b49cac03e71%40googlegroups.com.

J Decker

unread,
Aug 14, 2019, 6:39:05 AM8/14/19
to andro...@googlegroups.com
On Wed, Aug 14, 2019 at 3:36 AM J Decker <d3c...@gmail.com> wrote:
Well, I assumed that one library would be with all the other libraries so I have this bit of code


Err I guess it looks for the address of the routine itself instead of by name... the range that has the code is probably this library :) 

Alex Cohn

unread,
Aug 14, 2019, 9:52:32 AM8/14/19
to android-ndk
As Dan pointed out, on Android you should not worry about the paths to native libs. They can actually not exist on the file system (see https://developer.android.com/topic/performance/reduce-apk-size#extract-false). It's enough to call dlopen for the well-known name of the library. Furthermore, because the only legitimate way to deliver these libraries is to pack them inside your APK or bundle (possibly, one of the secondary APKs), in the prescribed path (/lib/{ABI}), there is no way you can use the path to do some logic (e.g. you cannot have a/libtest.so and b/libtest.so in your app),

Regards,
Alex

On Wednesday, August 14, 2019 at 1:36:33 PM UTC+3, J Decker wrote:
Well, I assumed that one library would be with all the other libraries so I have this bit of code


that reads /proc/self/maps and finds its own name (defined as part of the compile options TARGET_NAME as a string), and gets the path from that.  Works on any system that puts libraries in some place. 

Raj Shekar

unread,
Aug 19, 2019, 6:25:52 AM8/19/19
to andro...@googlegroups.com
Thanks again Dan, Alex, J Decker for the suggestions and for your time. It's good to know that we don't have to use absolute paths for dlopen calls. I'll try them out, thanks again for all your help. 

Thanks, 
Raj. 

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/8fc48a05-d272-4174-b700-cfafed027a6b%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages