How to change/specify a different CMake to be invoked by Android Studio/gradle?

[Eric Wing]

How can I change/specify a different CMake version/copy to be invoked?

I have two reasons for needing a different version:

(1) I need newer features in newer CMake versions than what is shipped or
(2) I am developing new features in CMake which means I need to use my
own build of it.


I don’t want to replace the SDK one on the file system. I am working
with other people, so I am looking for having a way to allow others to
do the same thing on their systems without much messing up their
standard development environment. So I am looking for some per-project
(non-global) way to override the CMake path. Perhaps something like
local.properties or some other configuration file could work (I’m open
to ideas).


Also, is the one that ships with Android a stock version of CMake or
has it been modified? (I see there is a toolchain file at the top. Is
that the only thing extra needed?)


Thanks,
Eric

[Dan Albert]

Right now you can't. Studio relies on a few changes to cmake that are only in our cmake distribution.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CA%2BQ62MCGNMJu_DOe0u6Focg06WP0vc_ev8LHtXm%3DXnW_80s6Vg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Eric Wing]

On 4/26/17, 'Dan Albert' via android-ndk <andro...@googlegroups.com> wrote:
> Right now you can't. Studio relies on a few changes to cmake that are only
> in our cmake distribution.

Is your CMake branch something I can pull and merge with?

[Dan Albert]

I don't think so, but I've CC'd Jomo, who would know for sure.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CA%2BQ62MAD9GvjVOY0QEcK04JtxH-N_RzhKL1ws_EtA%2BUqBkO7bA%40mail.gmail.com.

[Armen Avetisyan]

Hi there,

check my repo. I think I have what you need:

https://github.com/skanti/Android-Manual-Build-Command-Line

[Eric Wing]

On 4/27/17, Armen Avetisyan <armen.ave...@gmail.com> wrote:
> Hi there,
>
> check my repo. I think I have what you need:
>
> https://github.com/skanti/Android-Manual-Build-Command-Line

Thank you Armen, but I’m trying really hard to leverage the official
Android build chain. I’ve actually been doing something kind of like
your stuff for the past several years. I have been using a fork of the
CMake OpenCV Android toolchain along with a lot of other work/hacks
I’ve done to work with Android Studio/Gradle.

But it has been a ton of work to maintain, especially with all the
changes that have been going on. I’d like to utilize the official
stuff as much as possible and breaking away from my own stuff. And
there are new features I’d like to leverage and don’t want to
reimplement myself, such as the NDK-side debugging in Android Studio,
AAR libraries, and the NDK unified header changes.


So being able to leverage the official CMake support (which means
needing to merge my changes with the Android branch) is what I’m
really hoping for.

Thanks,
Eric

[Steven Winston]

So there are a couple of options for you.  The easiest involves writing your own intellij plugin.  The idea would be to create a new run configuration that uses your desired CMake distro from your desired location; then plugin all the rest of the Android plugin steps.  I've done this successfully with both a Lua Debugger and gtest build/configurations. You probably could get away with just switching the build tools path by abstracting the build process and inheriting from the gradle plugin; but I haven't really tried to hijack that portion of the build so I don't really know.

The next easiest way would be to grab the last source release for gradle that included cmake (it's been awhile so there's a lot of changes back).  From there, grab the corresponding CMake, create a list of changes and then try merging all those changes into the latest CMake.  The only problem is Google tends to not stand still so there's probably quite a few changes between their last public source release, and the one they currently are using in production so there's going to be changes that are missing.  This process is probably going to be very low success rate and impossible to keep up to date.  Thus if you have to go down a route of not using their CMake, I highly encourage you to in a very limited way look into the first option, Intellij plugin.

[Eric Wing]

On 4/28/17, Steven Winston <gpx...@gmail.com> wrote:
> So there are a couple of options for you. The easiest involves writing
> your own intellij plugin. The idea would be to create a new run
> configuration that uses your desired CMake distro from your desired
> location; then plugin all the rest of the Android plugin steps. I've done
> this successfully with both a Lua Debugger and gtest build/configurations.
> You probably could get away with just switching the build tools path by
> abstracting the build process and inheriting from the gradle plugin; but I
> haven't really tried to hijack that portion of the build so I don't really
> know.
> The next easiest way would be to grab the last source release for gradle
> that included cmake (it's been awhile so there's a lot of changes back).
> From there, grab the corresponding CMake, create a list of changes and
> then try merging all those changes into the latest CMake. The only problem
>
> is Google tends to not stand still so there's probably quite a few changes
> between their last public source release, and the one they currently are
> using in production so there's going to be changes that are missing. This
> process is probably going to be very low success rate and impossible to
> keep up to date. Thus if you have to go down a route of not using their
> CMake, I highly encourage you to in a very limited way look into the first
> option, Intellij plugin.
>

Hi Steven,

Your ideas sound pretty good.
I was told where Google’s CMake fork was and while messy, I think I
have a good chance of successfully merging with it.

So now I’m trying to figure out the Gradle plugin stuff. I don’t have
any familiarity with how this part works. But your idea of subclassing
the Google one, just enough to override the path to where CMake is
located, is really appealing. But I’m still not sure where to start.
I’m looking for two things.

1) Can you point me to some examples of how to subclass the Gradle
plugin, like how you did with your Lua Debugger?

2) I need find the official source code that controls invoking CMake
so I can understand what to override, but I don’t know where it
exists.


I tried the official process of using
repo init -u https://android.googlesource.com/platform/manifest -b
studio-master-dev
repo sync

But that took a 24 hour day to grab everything, and the first time
around, I missed the -b studio-master-dev (looking at a different set
of instructions). So the second time around took another full day as
it seemed to redownload everything, only to have it break with:

error: Cannot remove project "bionic": uncommitted changes are present
commit changes, then run sync again


(I can try another fresh run, but I’m taking a break of completely
saturating my internet connection so I can use it for other stuff.)



So I tried directly cloning:
https://android.googlesource.com/platform/tools/external/gradle/
and doing git checkout studio-master-dev
But everything inside seems to be a prebuilt binary, not source code.

I also looked at:
https://android.googlesource.com/platform/tools/gradle
but there didn’t seem to be much there that is relevant to this.


I also looked at what I believe is the official Gradle repository, but
I did not see any CMake stuff in there.
https://github.com/gradle/gradle.git


Any ideas of what I’m looking for and where it exists?


Thanks,
Eric

[Dan Albert]

I tried the official process of using
repo init -u https://android.googlesource.com/platform/manifest -b
studio-master-dev
repo sync
But that took a 24 hour day to grab everything

 

0_o

Do our docs not say to use `repo sync -c -j$SOME_NUMBER`? Which doc was this?

Adding in +jomof, who can hopefully point you to the right repositories for gradle/cmake things.

Thanks,
Eric

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+unsubscribe@googlegroups.com.
To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CA%2BQ62MD4h5Pn8aox4FoA%2BVN3MH1V4MhzwMiUqcizREF-Wn9YJg%40mail.gmail.com.

[Dan Albert]

Actually +jomof this time

[Jomo Fisher]

I don't know the answer about our CMake fork. Including Karthik, he may know.

The gradle plugin calls CMake executable in CmakeExternalNativeJsonGenerator.java. Warning, that code is in flux for 3.0.

[Eric Wing]

On 6/2/17, 'Dan Albert' via android-ndk <andro...@googlegroups.com> wrote:
>>
>> I tried the official process of using
>> repo init -u https://android.googlesource.com/platform/manifest -b
>> studio-master-dev
>> repo sync
>> But that took a 24 hour day to grab everything
>
>
> 0_o
>
> Do our docs not say to use `repo sync -c -j$SOME_NUMBER`? Which doc was
> this?
>
> Adding in +jomof, who can hopefully point you to the right repositories for
> gradle/cmake things.
>

There is a link on this page for "Follow the instructions here to get
checkout the source code":
http://tools.android.com/build/gradleplugin

which leads to this page:
http://tools.android.com/build

which has this block:

$ mkdir studio-master-dev
$ cd studio-master-dev
$ repo init -u https://android.googlesource.com/platform/manifest -b
studio-master-dev
$ repo sync

Thanks,
Eric

[Eric Wing]

On 6/2/17, 'Jomo Fisher' via android-ndk <andro...@googlegroups.com> wrote:
> I don't know the answer about our CMake fork. Including Karthik, he may
> know.
> The gradle plugin calls CMake executable
> in CmakeExternalNativeJsonGenerator.java. Warning, that code is in flux for
> 3.0.
>

Thanks.
Looking at that file, I found:

@NonNull
private File getCmakeFolder() {
return new File(getSdkFolder(), "cmake");
}


I need a way to override this, to make Android Studio look for a CMake
located somewhere that I specify, which will most likely not be in the
SdkFolder.

Can you recommend an approach to do this? The original idea was to
write a Gradle plugin that inherits from it to override the path, but
then leverage everything else in the base. But looking at this, I am
less and less certain how to actually make that work.



And since this is in flux, may I make a feature request? Would you add
a new "CMakeOptions" parameter to your Gradle plugin to allow
overriding the path to CMake.

Currently CmakeOptions only has one parameter, "paths", which points
to the CMakeLists.txt.

I propose 2 additional optional parameters. One that specifies the
path to CMake, e.g "cmakeFolder". And a second that optionally allows
specifying a toolchain. (I expect that by default, when not explicitly
specifying the toolchain, it will look in the
cmakeFolder/android.toolchain.cmake. If cmakeFolder is not specified,
then that folder is exactly what it does today.)

These would obviously be for advanced users, but I think they are
still useful. Besides people like me who are developing new features
for CMake, there have already been people needing newer features in
CMake from 3.7/3.8 which are not in your 3.6 branch. This would give
those people a fighting chance to try updating/merging themselves
while waiting for an update.

Thanks,
Eric

[Steven Winston]

I'm not sure I'm the most knowledgeable person to offer advice, Jomo, Dan and others know way more than I do about all of this.  However, I wanted to clear up something really quickly in my original suggestion.

Android Studio is based upon Intellij and the build process involves several technologies: Intellij, Gradle, CMake, Ninja are the ones relevant to the pipeline here.  The two places capable of getting plugins are Intellij and Gradle ahead of CMake.

What happens is your project has a Run Configuration, that Run Configuration is defined by the Android Plugin to Intellij.  When you try to build, Intellij calls it an AnAction command that tells the run configuration to launch the executable.  You can capture this with your own AnAction Intellij task that places a button, menu option whatever; or create your own Run Configuration inheriting from the Android Run Configuration.  As a plugin that initiates the build process, you can gain access to the build tools that Android Plugin is using and make your updates there prior to launching a build event.  I'm not certain that the path to CMake executable is one of the public accessors that Android plugin uses, but you have direct access at this point to create your own thing.  Here's the resource to get more details: https://intellij-support.jetbrains.com/hc/en-us/community/topics/200366979-IntelliJ-IDEA-Open-API-and-Plugin-Development

Gradle however, has its' own plugin system that is completely separate from Intellij (an Android Studio plugin would have nothing to do with Gradle plugin).  All Intellij does is to launch gradle as an executable.  A Gradle plugin would be something that modifies your build tasks.  This you can do too:  https://docs.gradle.org/3.3/userguide/custom_plugins.html  If you go this route, you're going to want to look at modifying the *ExternalNativeJsonGenerator task  You should be able to create your own there and do this as a direct Gradle plugin.

Either way a bit of a warning: This might be a bit of a challenge for a first plugin.  It is certainly possible to do, but I do want to warn that there be subtle challenges within this endeavor that I might not even be aware of.

[Geordie J]

Just wondering- this post is from 2017 but using a new CMake version is still something that'd be quite useful for various reasons. Has there been any improvements to the workflow since the OP?

Thanks,
Geordie

[Dan Albert]

https://developer.android.com/studio/projects/add-native-code.html#use_a_custom_cmake_version

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.

To post to this group, send email to andro...@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/6dc16df1-f1b7-4e52-868f-90326d19739b%40googlegroups.com.

[Geordie J]

This is awesome news, thank you! (And sorry, I could have found this myself with some googling but I expected the answer to be more difficult). PS love your avatar haha :)