ndkBuild and CMake in one project but dependencies don't fire, libraries not found

[Andi McClure]

Short version: I have constructed a gradle NDK project where the main project uses ndkBuild and a subproject uses CMake. I want the CMake project to produce a library the ndkBuild project links, but the dependency to the CMake project is not being applied, the library is not found by the main project, and I do not know how to specify to the main project the filename it’s supposed to be extracting from the CMake project.

Context: I have an existing video game which I am porting to the Oculus Mobile SDK 1.16.0 https://developer.oculus.com/downloads/package/oculus-mobile-sdk/ . This SDK contains libraries and sample code, both of which are built using Gradle and ndkBuild. The SDK’s ndkBuild scripts for the sample projects are pretty complicated and I do not know enough about how Oculus works to port them to CMake myself. Meanwhile the project I am porting is based on CMake and would be *very* difficult or impossible to port to ndkBuild, since it includes CMake builds of very complex projects such as Assimp and LuaJIT. For this reason, it is necessary I use both CMake and ndkBuild in this single project.

Mixing CMake and ndkBuild in one build.gradle turns out to be not allowed ( it produces an error https://android.googlesource.com/platform/tools/base/+/studio-2.2-preview3/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/ExternalNativeBuildTaskUtils.java#179 ). But what I *can* do is create a project and a subproject, and have one use ndkBuild and the other CMake.

Currently my project is based on the VrCubeWorld_Framework example from the Oculus Mobile SDK. Its  dependencies {} section depends on five projects, four Oculus libraries (:VrAppFramework:Projects:Android, etc) and one mine (“implementation project(':cmakelib’)”). :cmakelib is a project at the top of my repository. Its build.gradle is simple (attached). It includes a CMakeLists.exe which builds a library named “lovr”, created with CMake add_library(lovr SHARED ${SOURCES}). I am using command line gradlew to compile because I could not get Android Studio to create a project without errors.

Some of what is happening makes sense to me:

1. I can build cmakelib by itself with "gradlew installDebug” and it builds and uploads an apk that is (unsurprisingly) not a valid Android program, but if I unzip it, I can see it contains a file in lib/[ARCH] named liblovrd.so.

2. I can build VrCubeWorld_Framework with “gradlew installDebug” and it builds and uploads an apk which is a valid Android program I can execute on my Oculus Go, *and* if I unzip the APK I can see it contains that same lib/[ARCH]/liblovrd.so.

Some of what is happening does *not* make sense to me, and is very bad:

1. Although builds of VrCubeWorld_Framework are triggering builds of cmakelib, as if it were a dependency, the builds of cmakelib are not occurring “before” the build of VrCubeWorld_Framework, as you would expect for a dependency.

What I mean by this: Say that I introduce a failure into cmakelib, for example I add “message(FATAL_ERROR test)” to the CMakeLists.txt. When I installDebug the VRCubeWorld_Framework project, it will fail with the expected message. But let’s say that I introduce failures to cmakelib and VRCubeWorld_Framework at once (I put an error in the source file listed in VRCubeWorld_Framework’s jni/Android.mk). VRCubeWorld_Framework will be built first, and will print its error, and the build will halt, and cmakelib will not be attempted. It appears VRCubeWorld_Framework compiles first and cmakelib after.

This is especially a problem because:

2. VRCubeWorld_Framework, when its jni/Android.mk is being built, cannot find liblovrd.so, and the symbols in liblovrd.so are not visible to its source files. There is a symbol bridgeLovrInit(); in liblovrd.so. If I add “lovr” to LOCAL_SHARED_LIBRARIES in VRCubeWorld_Framework’s jni/Android.mk, and add a call to bridgeLovrInit() in the VRCubeWorld_Framework’s main cpp file, then build, I get these errors at the end of the gradlew output:

  /Users/mcc/Library/Android/sdk/ndk-bundle/build/core/build-binary.mk:688: Android NDK: Module vrcubeworld depends on undefined modules: lovr    

  [arm64-v8a] Compile++      : vrcubeworld <= VrCubeWorld_Framework.cpp

  [arm64-v8a] Prebuilt       : libvrapi.so <= /Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/jni/../../../../../VrApi/Projects/AndroidPrebuilt/jni/../../../Libs/Android/arm64-v8a/Debug/

  [arm64-v8a] SharedLibrary  : libvrcubeworld.so

  /Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/build/intermediates/ndkBuild/debug/obj/local/arm64-v8a/objs-debug/vrcubeworld/__/__/__/Src/VrCubeWorld_Framework.o: In function `OVR::VrCubeWorld::EnteredVrMode(OVR::ovrIntentType, char const*, char const*, char const*)':

  /Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/jni/../../../Src/VrCubeWorld_Framework.cpp:116: undefined reference to `bridgeLovrInit()'

  clang++: error: linker command failed with exit code 1 (use -v to see invocation)

Notice the warning on the first line and the error on the final line.

Other things which do not work: If I name lovr in the LOCAL_SHARED_LIBRARIES line to “liblovr”, it fails the same way, with the “undefined module lovr” error; if I say “liblovrd” it gives me an error of “undefined module lovrd”. If I construct “lovr” as a static library (I would actually rather it be static) it can’t find it like that either.

What am I supposed to do here? The other shared libraries included by Android.mk are also coming out of subprojects, but those are all ndkBuild based subprojects. It feels like some kind of magic is allowing the ndkBuild projects to see each other which is not working when the subproject is cmake. How are projects and subprojects in gradle supposed to communicate outputs/inputs to each other?

[Andi McClure]

As a minor update, I have found I am able to include my CMake-built code into the ndkBuild project if I do two things: First, I add this to my Android.mk ... 

include $(CLEAR_VARS)

ifdef NDK_DEBUG

LOVR_LIB_SUFFIX=d

else

LOVR_LIB_SUFFIX=

endif

LOCAL_MODULE := lovr

LOCAL_SRC_FILES := ../../../../../cmakelib/build/intermediates/cmake/debug/obj/$(TARGET_ARCH_ABI)/liblovr$(LOVR_LIB_SUFFIX).so

include $(PREBUILT_SHARED_LIBRARY)

... and two, I always run `gradlew build` on the cmakelib subproject before I run `gradlew installDebug` on the main project. The Android.mk change means that when the code of the main project is built it can link the symbols in liblovrd.so, because it views the liblovrd.so in the CMake intermediate build products directory as a prebuilt library; then running the CMake build first means that the .so is there and has the correct symbols even though gradle triggers its own dependency of the CMake build after the ndkBuild build. (There are several .so files that cmakelib generates that liblovrd.so depends on, but which are not directly referenced by the main project; for whatever reason, the project manages to correctly copy these additional .sos into the APK even though I don't cite them in the Android.mk).

This is all *really scary*. This approach is brittle, sketchy, requires this two-step build that if I forget to do it in the right order can lead to strange errors, and depends on the paths and names of temporary files created by CMake remaining stable in the future. I have doubts I will be able to make a static library, or have liblovrd.so link back into the main ndkBuild project, with this solution. I am at least temporarily unblocked such I can start writing code instead of beating my head against CMake, but I really hope I won't need to ship a product with this approach. It seems like all the information to make this work more properly is available to gradle, it's just somehow not being communicated to ndkBuild that the CMake subproject is a "module". Is there really no way to do this with the current system?

[Steven Winston]

This is admittedly an area that Android studio could use a bit of work in. There's two suggestions I have:
Use cmake and create a parent project in cmake to build everything. This would give you externalproject* family of cmake functions that correctly handle things in a sane way.
This has the advantage of having a single point where your build goes between Gradle and the native build system.
Or you could go down the route of using Gradle to invoke an "install" step for your module and place the Libs in the right place. You can have the install step get invoked on build automatically.
The advantage here is the ide feels a bit better and you can take advantage of some module logic that's a bit more difficult to get access to.
I personally recommend the first path as the preview AS has some features that makes this easier.

[Andi McClure]

One update to my previous post— I have figured out my "problem 1", where the dependencies were not running in the correct order— I reordered the lines in my build.gradle and this problem fixed itself completely. That was actually the larger of the two problems, so that's good.

Steven, thank you for the advice-- there's a couple issues.

As for using externalproject in cmake— in this *specific* case I need to invoke cmake from ndkbuild rather than the other way around. But if I did use externalproject, how would cmake invoke ndkbuild? Is the idea externalproject would invoke the gradlew executable?

For the other suggestion— You're right, I *could* get either cmake or gradle to install the libs, or "install" them to a known local directory, and that would definitely be less scary than linking into the build intermediates directory. A weird thing about that solution though is that the installed-intermediates directory would only be used so the ndkBuild step could accurately check symbols against the cmake outputs! Gradle is actually copying the cmake outputs into the apk, it just doesn't seem to be making them visible to the ndkbuild step. I suppose at least it would work, though.

[Alex Cohn]

The main drawback of the externalproject approach is that this way Android Studio will not sniff correctly the ndk-build files for code navigation, completion, etc. If some library is stable and you don't need to open its sources in AS, it's actually recomended to move it out of code inspection: AS does not yet handle big C++ projects yet. We keep our CMake builds for stable libraries 'hidden' from Gradle, otherwise the IDE becomes unuseably slow.

If libraries that are in active development (and may be edited and debugged in AS) require both CMake and ndk-build, there is no alternative to keep them is separate sub-projects (a.k.a. Modules), but there is no 'build tool agnostic' way to get the two understand one another, e.g. let ndk-build pick the location of the built dependency from CMake.

BR,

Alex

[Alex Cohn]

There is a nasty side-effect of using PREBUILT_SHARED_LIBRARY. This binary will be copied to your install directory, and eventually to the APK. This would be nice, except Gradle and CMake already take care of that, looking at this binary as a deliverable from the other Module.

The easy workaround is to use PREBUILT_STATIC_LIBRARY instead.

BR,

Alex