Gradle + CMake + Android Studio Slowness Problems

[Nick Weihs]

I have a rather large CMake setup for my game engine that takes a minute or so to configure due to all the feature detection CMake needs to do.  For other platforms, this isn't really a big deal since I generally only need to configure once even if I'm adding new files.  I've set up a new Android Studio project that links to my existing CMakeLists.txt to build a shared library, and it does work eventually, however it's taking a long long time.  For whatever reason, gradle/Android Studio is running the CMake configure about 10 times back to back before it even starts compiling anything.  So just making a minute change to CMakeLists.txt or one of it's sub projects can eat up about 15 minutes of time just configuring over and over again.

My theory is that gradle is configuring CMake for every build task in the default gradle setup, but I have no knowledge of how to tell if that's the case let alone fix it.

Is anyone else seeing this problem or know how to fix it?

[Dan Albert]

The gradle sync step runs cmake so it can learn things like what compilation flags you're using aiui (needed for IDE features). Right now it does that for all of your supported ABIs, for both debug and release. This is going to get fixed in the future so it only syncs one config instead. I'm not sure, but you may be able to mitigate this a bit by changing your abiFilters to only a single target during development and then put it back once you're ready to test other ABIs.

FYI, this only results in abysmally slow behavior on Windows because CreateProcess takes so much longer than fork (at least, that's what appears to be the difference). If it's an option for you, using Linux or Mac (even in a VM) would speed things up significantly.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/ada0bf2b-eb4a-4661-9889-0b4eb8dde0bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Henrik Rydgård]

Is there an ETA on the fix to only gradle sync one config, or at least an open issue to track? It's not only for every ABI and debug/release but also for every flavor so there's a combinatorial explosion that in my case results in a 10 minute gradle sync, which is rather annoying when trying to use git bisect when there are changes to the CMakeLists.txt... 

On Sunday, November 12, 2017 at 5:07:10 AM UTC+1, Dan Albert wrote:

The gradle sync step runs cmake so it can learn things like what compilation flags you're using aiui (needed for IDE features). Right now it does that for all of your supported ABIs, for both debug and release. This is going to get fixed in the future so it only syncs one config instead. I'm not sure, but you may be able to mitigate this a bit by changing your abiFilters to only a single target during development and then put it back once you're ready to test other ABIs.

FYI, this only results in abysmally slow behavior on Windows because CreateProcess takes so much longer than fork (at least, that's what appears to be the difference). If it's an option for you, using Linux or Mac (even in a VM) would speed things up significantly.

On Fri, Nov 10, 2017 at 5:41 PM, Nick Weihs <nick....@gmail.com> wrote:

I have a rather large CMake setup for my game engine that takes a minute or so to configure due to all the feature detection CMake needs to do.  For other platforms, this isn't really a big deal since I generally only need to configure once even if I'm adding new files.  I've set up a new Android Studio project that links to my existing CMakeLists.txt to build a shared library, and it does work eventually, however it's taking a long long time.  For whatever reason, gradle/Android Studio is running the CMake configure about 10 times back to back before it even starts compiling anything.  So just making a minute change to CMakeLists.txt or one of it's sub projects can eat up about 15 minutes of time just configuring over and over again.

My theory is that gradle is configuring CMake for every build task in the default gradle setup, but I have no knowledge of how to tell if that's the case let alone fix it.

Is anyone else seeing this problem or know how to fix it?

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.

[Dan Albert]

Not sure if there's a bug open. +jomof would know.

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/49ee776f-28fe-48b6-9334-c519ee38c802%40googlegroups.com.

[Jomo Fisher]

>> So just making a minute change to CMakeLists.txt or one of it's sub projects can eat up about 15 minutes of time just configuring over and over again.

Starting in AS 3.1, which is in beta right now, we try to be smarter about when we regenerate the CMake project. We try to make better use of CMake's incremental project update.. This work is covered by https://issuetracker.google.com/69408798

Single-config sync may make it into AS 3.2, but single ABI sync won't. However, once you have single config you could set up a config just for development that builds just the ABI you need for development.

[Alex Cohn]

We use the following lifehack. You may find it useful, too:

We wrap our CMakeLists.txt with the following clause:

if (CMAKE_BUILD_TYPE STREQUAL "Release")
  add_custom_target(dummy)
else()
  …
endif()

Now our Sync does real work only for Debug configuration. Also, don't underestimate abiFIlters - for Debug, you usually only need one flavor there, x86 if you work with emulator.

BR,

Alex

[Steven Winston]

FWIW:

A trick I've found quite useful (speak of life hacks) that solves the speed of build problem caused by ABI * productFlavor * buildType rebuilds is to solve it with CMake.

The reason there's an issue is the build folder is different for each ABI * productFlavor * buildType.  Thus, tell CMake to build to a common folder:

i.e.

set_target_properties( targets...

    PROPERTIES

    ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BIN_DIR}<pathToLibs>"

    LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BIN_DIR}<pathToLibs>"

    RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BIN_DIR}<pathToLibs>"

)

set(CMAKE_BIN_DIR <PathToFolder outside of .externalNativeBuild>)

By setting the output directories at the start, you stand a good chance of having the packaging system be tricked into believing that the .so files are in the proper locations.  I'm avoiding copying pasting code from internal source and writing this off the top of my head so please forgive the "you must do your own homework" style suggestion here.