Compilation of Kotlin in Gradle ignores project build order

[Przemysław Wesołek]

Hello,

I have problem with compiling Kotlin sources in bnd workspace and I'm not sure if the problem lies in Kotlin or bnd Gradle plugin, or should I modify my build.gradle to fit the situation.

The example project is on https://github.com/jest/test-bnd-gradle-kotlin-nodepends-bug

I have two bundles, b.dependant.api and a.depending.provider. A depends on B, so B must be build before A is compiled. It works correctly, when only Java sources are present, but as soon as I add Kotlin file and configure Gradle to compile it, Kotlin compilation ignores A->B dependency and A compilation results in error:

$ ./gradlew clean
Starting a Gradle Daemon, 1 stopped Daemon could not be reused, use --status for details
:a.depending.provider:clean
:b.dependant.api:clean

BUILD SUCCESSFUL

Total time: 5.413 secs
$ ./gradlew jar
:a.depending.provider:compileKotlin
e: /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src/a/depending/provider/DependingExampleKotlin.kt: (3, 8): Unresolved reference: b
e: /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src/a/depending/provider/DependingExampleKotlin.kt: (6, 30): Unresolved reference: Example
:a.depending.provider:compileKotlin FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':a.depending.provider:compileKotlin'.
> Compilation error. See log for more details

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 2.814 secs

When I manually build B first, it works fine:

$ ./gradlew clean
:a.depending.provider:clean
:b.dependant.api:clean

BUILD SUCCESSFUL

Total time: 1.037 secs
$ ./gradlew b.dependant.api:jar
:b.dependant.api:compileJava
:b.dependant.api:processResources UP-TO-DATE
:b.dependant.api:classes
:b.dependant.api:jar

BUILD SUCCESSFUL

Total time: 1.629 secs
$ ./gradlew jar                                                                                                                  
:a.depending.provider:compileKotlin
w: /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src/a/depending/provider/DependingExampleKotlin.kt: (6, 18): Parameter 'dependency' is never used
:b.dependant.api:compileJava UP-TO-DATE
:b.dependant.api:processResources UP-TO-DATE
:b.dependant.api:classes UP-TO-DATE
:b.dependant.api:jar UP-TO-DATE
:b.dependant.api:assemble UP-TO-DATE
:a.depending.provider:compileJava
:a.depending.provider:copyMainKotlinClasses
:a.depending.provider:processResources UP-TO-DATE
:a.depending.provider:classes
:a.depending.provider:jar

BUILD SUCCESSFUL

Total time: 1.862 secs

Building with --debug shows that the build path includes B's bundle only when it was proviously built (see "compiling with classpath" in the last lines):

With errors:

13:48:14.226 [DEBUG] [org.gradle.api.Task] [KOTLIN] all kotlin sources: a.depending.provider/src/a/depending/provider/DependingExampleKotlin.kt
13:48:14.227 [DEBUG] [org.gradle.api.Task] [KOTLIN] compileKotlin destinationDir = /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/generated/kotlin-classes/main
13:48:14.227 [DEBUG] [org.gradle.api.Task] [KOTLIN] compileKotlin source roots: [
        /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src,
        /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src/main/kotlin]
13:48:14.228 [DEBUG] [org.gradle.api.Task] [KOTLIN] compileKotlin java source roots: [
        /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src]
13:48:14.228 [DEBUG] [org.gradle.api.Task] [KOTLIN] Removing all kotlin classes in /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/generated/kotlin-classes/main
13:48:14.231 [DEBUG] [org.gradle.api.Task] [KOTLIN] compiling with args: [-no-stdlib, -no-reflect, -module, /tmp/kjps4283698478727396698adependingprovider_main.script.xml, -module-name, a.depending.provider_main, -jvm-target, 1.6, -language-version, 1.0, -api-version, 1.0]
13:48:14.231 [DEBUG] [org.gradle.api.Task] [KOTLIN] compiling with classpath: /home/jest/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-runtime/1.0.5/67d6f90bf8ed20fcdc78a23fcdc6804749d24498/kotlin-runtime-1.0.5.jar, /home/jest/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.0.5/5784635914dadf0635e7bf3f28f1711317e1433a/kotlin-stdlib-1.0.5.jar

Without errors:

13:49:55.426 [DEBUG] [org.gradle.api.Task] [KOTLIN] all kotlin sources: a.depending.provider/src/a/depending/provider/DependingExampleKotlin.kt
13:49:55.426 [DEBUG] [org.gradle.api.Task] [KOTLIN] compileKotlin destinationDir = /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/generated/kotlin-classes/main
13:49:55.427 [DEBUG] [org.gradle.api.Task] [KOTLIN] compileKotlin source roots: [
        /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src,
        /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src/main/kotlin]
13:49:55.428 [DEBUG] [org.gradle.api.Task] [KOTLIN] compileKotlin java source roots: [
        /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/src]
13:49:55.428 [DEBUG] [org.gradle.api.Task] [KOTLIN] Removing all kotlin classes in /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/a.depending.provider/generated/kotlin-classes/main
13:49:55.433 [DEBUG] [org.gradle.api.Task] [KOTLIN] compiling with args: [-no-stdlib, -no-reflect, -module, /tmp/kjps8502199755777343608adependingprovider_main.script.xml, -module-name, a.depending.provider_main, -jvm-target, 1.6, -language-version, 1.0, -api-version, 1.0]
13:49:55.434 [DEBUG] [org.gradle.api.Task] [KOTLIN] compiling with classpath: /home/jest/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-runtime/1.0.5/67d6f90bf8ed20fcdc78a23fcdc6804749d24498/kotlin-runtime-1.0.5.jar, /home/jest/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.0.5/5784635914dadf0635e7bf3f28f1711317e1433a/kotlin-stdlib-1.0.5.jar, /home/jest/work/eclipses-ws/gradle_bnd_kotlin_nodepends_bug/b.dependant.api/generated/b.dependant.api.jar

[BJ Hargrave]

The Bnd Gradle plugin adds function to the Gradle Java Plugin. It knows nothing about Kotlin source and the Kotlin plugin.

The Bnd Gradle plugin uses Bnd to read and process the cnf folder and the project's bnd.bnd files. The Bnd Gradle plugin then uses the information processed by Bnd to set up the gradle build (sourceSets, compile paths, task dependencies). Since the Bnd Gradle plugin is unaware of Kotlin (and many other things), it does not establish the necessary relationships to build Kotlin code. You have a build.gradle file for the Kotlin project to set up the gradle plugin and add a compile dependency. You will also need to set up the necessary task dependencies as well. The compileKotlin task(s) needs to make dependent on the proper task(s) to ensure that they are executed first. In your example project, the :a.depending.provider:compileKotlin task will need to be dependent on the :b.dependant.api:assemble task. See https://github.com/bndtools/bnd/blob/master/biz.aQute.bnd.gradle/src/aQute/bnd/gradle/BndPlugin.groovy#L592-L602 where the Bnd Gradle plugin does this for java project dependencies.

--
You received this message because you are subscribed to the Google Groups "bndtools-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

BJ

[Przemysław Wesołek]

Thanks BJ, something like this in build.gradle solves the problem:

bnd.project.getDependson()*.getName().each { dependency ->
    compileKotlin.dependsOn(":${dependency}:assemble")
}

Przemek

[BJ Hargrave]

I did some more digging and it is much simpler. Just make the project's compileKotlin task depend upon the project's compileJava task. This is what the groovy plugin does and also what kotlin's own android gradle plugin does. Not sure why they don't so this for the normal kotlin plugin.

The compileJava task already depends upon upon the proper dependencies.

I could update the Bnd Gradle plugin to probe for a compileKotlin task, and if present, make it depend upon the compileJava task.

[Przemysław Wesołek]

This is not that simple, I tried. :)  Kotlin plugin also modifies compileJava by adding a dependency on compileKotlin. So adding compileKotlin->compileJava by myself creates a cycle, which is forbidden in Gradle.

Of course, I could remove the Java->Kotlin compilation dependency, but then it is no longer "simple", and it is a bit fragile, e.g. when you have in one project mixed Kotlin+Java sources, and Kotlin classes indeed depend on Java classes, you have no way but to compile it Java-first.

I ended up with the following modified build.gradle in the root project (i.e. Bndtools workspace dir):

/* Add bnd gradle plugin as a script dependency */
buildscript {
  ext.kotlin_version = '1.0.5'
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath bndPlugin
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
  }
}

// ... a few lines later ...

/* Configure the subprojects */
subprojects {
  def bndProject = bndWorkspace.getProject(name)
  if (bndProject != null) {
    plugins.apply 'biz.aQute.bnd'
    plugins.withType(org.jetbrains.kotlin.gradle.plugin.KotlinPluginWrapper) {
      bndProject.getDependson()*.getName().each { dependency ->

        compileKotlin.dependsOn(":${dependency}:assemble")
      }

      repositories {
          mavenCentral()
      }
      dependencies {
          compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
      }
    }
  }
}

and then in my Kotlin project's build.gradle I only need one line:

apply plugin: "kotlin"

Przemek

[BJ Hargrave]

Hmm. I guess  there is the question of whether the other JVM language's (Kotlin, Groovy) compile task should depend on compileJava or compileJava should depend upon the other JVM language's compile task. For Groovy, compileGroovy dependsOn compileJava. So applying the 'groovy' plugin works great with the Bnd Gradle Plugin (we use it on the Bnd build to make the Bnd Gradle plugin). But for Kotlin, it seems compileJava dependsOn compileKotlin (which seems wrong to me.)

Sigh... so much for consistency. The Groovy plugin clearly documents that compileGroovy dependsOn compileJava [1]. I cannot find any such clear documentation for Kotlin plugin. So it means that what you did is what is needed. If there was better, more precise documentation for the Kotlin plugin, I could see making changes to the Bnd Gradle plugin to support Kotlin. But without that doc, it is risky since the Kotlin plugin may change. I guess I would feel better if the Kotlin plugin was part of the Gradle distribution like the Groovy plugin.

[1] https://docs.gradle.org/current/userguide/groovy_plugin.html#sec:groovy_tasks

[Przemysław Wesołek]

As for Kotlin support... let's say that the large port of its community in IntelliJ users, and as long as they are happy, no big developments in other areas happen. I follow release notes for Kotlin for quite some time and when they say "IDE improvements", they mean "IntelliJ improvements". So: no big Eclipse roadblocks removal, no official Gradle plugin in the repository, etc. Even the classname of the KotlinPlugin which I had to put in build.gradle is nowhere documented, it's hacked in by me with trial-and-error method...

So I agree with you that adding Bnd Gradle plugin support for Kotlin is rather a bleeding edge. On the other hand, if so little is needed as in my case, is it really so risky?

[BJ Hargrave]

> On the other hand, if so little is needed as in my case, is it really so risky?

Well until there is documented API for the kotlin gradle plugin, depending upon the name of types and tasks is like depending upon internal information. It is subject to change without notice. Perhaps the answer for now is for a github wiki page describing what you have learned so that others can benefit. I'll put one together.

[BJ Hargrave]

So I put up this wiki page: https://github.com/bndtools/bnd/wiki/Using-Kotlin-with-the-Bnd-Gradle-Plugin-for-Workspace-Builds

--

BJ

[Przemysław Wesołek]

Thanks once again!

Przemek