Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Gradle JavaCpp - Tests in same gradle-sub-project

52 views
Skip to first unread message

Thomas Natschläger

unread,
Jul 30, 2021, 6:54:06 AM7/30/21
to javacpp
Hi!

Thank you for making https://github.com/bytedeco/gradle-javacpp available.
I got it it to work for a gradle project where I have to sub-projects:

mylib: this builds the native library using gradles cpp-library plugin
myjni: this uses gradle-javacpp to generate the Java JNI interface to it.

myjni also has Unit Tests written in Java to see if everything works out. I can actually run thos tests nicely and working via `gradlew myjin:test` on my windows machine.

However, this has to be cross-plattform and I know trying to get the thing working on linux too. And here I'm struggling to get `gradlew myjin:test`work. It always complains that the native libraries (jnimylib.so and mylib.so) can not be found.

The difference I figured out so far is, that under Windows (where I'm using mingw64) the natie libraries (*.dll) go into myjni.jar and not into myjni-windows-x86_64.jar, while on Linux the native libraries (*.so) go to myjni-linux-x86_64.jar. May this actually be the reason why they are found under windows and not under Linux?

So my questions are:
a) Is it intended that under window the *.dll do not show up in myjni-window-x86_64-mingw.jar or I'm doing something wring here?
b) How to get myjni-linux-x86_64.jar on the class-path such thet the tests can be run via `gradlew myjin:test

Thank you!

Kr

  -Thomas (Natschläger)

PS: For reference here is my gradle.build file for myjni sub-project:

plugins {
    id 'java-library'
    id 'java-test-fixtures'
    id 'org.bytedeco.gradle-javacpp-build' version "1.5.5"
}

dependencies {

    // the java bindings depend on the native library
    runtimeOnly project(path: ":mylib", configuration: 'sharedLibraryRelease')

    // we need JavaCpp :)
    api "org.bytedeco:javacpp:1.5.5"

    // Implementation for test fixtures
    testFixturesImplementation "org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}"
    testFixturesImplementation "org.junit.jupiter:junit-jupiter-params:${junitJupiterVersion}"

    testImplementation 'org.apache.commons:commons-math3:3.6.1'

    testImplementation "org.bytedeco:javacpp:1.5.5"
    testRuntimeOnly "org.bytedeco:javacpp:1.5.5:linux-x86_64"
}

/**
 * Java CPP specific stuff follows
 */
ext {
    println "javacppPlatform: " + javacppPlatform
}

// This setting are still a hack.
// One should not use such direct hardcoded references to files or directories of other submodules.
tasks.withType(org.bytedeco.gradle.javacpp.BuildTask) {
    // tell JavaCpp where the include files referenced in the interface configuration class
    includePath = [
        "$projectDir/../mylib/src/main/include"
    ]

    // tell JavaCpp in which folders to look for the libraries to link
    linkPath = [
        "$projectDir/../mylib/build/lib/main/release"
    ]
}


javacppBuildCommand {
    // building of the native part of mylib is done in the mylib project.
    // Hence, it is left empty intentionally.
}

javacppBuildParser {
    // this is the java classes which define the JNI interface generation
    classOrPackageNames = ['org.myjni.presets.*']
}

javacppBuildCompiler {
    // is set in all examples to true
    copyLibs = true
    deleteJniFiles = false
}

sourcesJar {
    duplicatesStrategy = DuplicatesStrategy.INCLUDE
}



Samuel Audet

unread,
Jul 30, 2021, 7:08:56 AM7/30/21
to javacpp...@googlegroups.com, Thomas Natschläger
Generally speaking, you'll need myjni-linux-x86_64.jar in your class path for the code to run on Linux, yes,
but the output of each plugin automatically gets added to the class path of the next plugins, so it usually just works, for example:
But this is with everything in a single project. I don't understand why you're trying to split the build in 2 projects.
Could you elaborate on the kinds of limitations that you're facing that prevents you to do it the easy way?

The DLLs on Windows should end up in a separate artifact as well, but it's only going to work if its directory is equal to javacppPlatform:
Though you can play with the tasks to get more custom behavior, I would again recommend against doing things the hard way...

Samuel

Thomas Natschläger

unread,
Aug 3, 2021, 10:34:24 AM8/3/21
to javacpp
Thank you Samuel for your prompt reply!

I tried a bit harder (I guess I'm a gradle novice) and got everything in one sub-project to work, mostly. I still find the following issues:

1) medium: On windows the DLLs are not ending up in myjni-windows-x86_64-mingw.jar but in  myjni-windows.jar (is this an issue with mingw-64?).
3) medium: With gradle 7.1.1 I get a lot of warnings like:

Execution optimizations have been disabled for task ':mylib:linkDebug' to ensure correctness due to the following reasons:
  - Gradle detected a problem with the following location: 'C:\workspaces\mylib\build\lib\main\debug'. Reason: Task ':mylib:javacppBuildCommand' uses this output of task ':mylib:linkDebug' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.1.1/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: 'C:\workspaces\mylib\build\lib\main\debug\mylib.dll'. Reason: Task ':mylib:javacppBuildCommand' uses this output of task ':mylib:linkDebug' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.1.1/userguide/validation_problems.html#implicit_dependency for more details about this problem.
Gradle detected a problem with the following location: 'C:\workspaces\mylib\build\lib\main\debug'. Reason: Task ':mylib:javacppBuildCommand' uses this output of task ':mylib:linkDebug' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.1.1/userguide/validation_problems.html#implicit_dependency for more details about this problem. This behaviour has been deprecated and is scheduled to be removed in Gradle 8.0. Execution optimizations are disabled to ensure correctness. See https://docs.gradle.org/7.1.1/userguide/more_about_tasks.html#sec:up_to_date_checks for more details.
Gradle detected a problem with the following location: 'C:\workspaces\mylib\build\lib\main\debug\mylib.dll'. Reason: Task ':mylib:javacppBuildCommand' uses this output of task ':mylib:linkDebug' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.1.1/userguide/validation_problems.html#implicit_dependency for more details about this problem. This behaviour has been deprecated and is scheduled to be removed in Gradle 8.0. Execution optimizations are disabled to ensure correctness. See https://docs.gradle.org/7.1.1/userguide/more_about_tasks.html#sec:up_to_date_checks for more details.

3) minor: On Linux I still had to add "$buildDir/libs/" + javacppJar.archiveName to the test classpath otherwise it would not find the *.so during the test runs.

Any more thoughts on this?
Thanks and Kr

-Thomas

Again for reference my now combine build.gradle file:

plugins {
    // This plugin allows to build C++/C libraries
    id 'cpp-library'

    // here comes the java JNI related plugins
    id 'java-library'
    id 'org.bytedeco.gradle-javacpp-build' version "1.5.5"
}

//////////////////////////////////////////////////////////////////////////////////////
// Build the native library
//////////////////////////////////////////////////////////////////////////////////////


library {
    // name of the library artifact
    baseName = "mylib"

    // where are the sources: all *.cpp (not *.c) files in that folder are compiled and linked as part of the library.
    source.from file('src/main/cpp')

    // header files not to be published
    privateHeaders.from file('src/main/cpp/mylib')

    // head files to develop against the library
    publicHeaders.from file('src/main/include')

    // we want a shared library
    linkage = [Linkage.SHARED]
}


tasks.withType(CppCompile).configureEach {

    // this macro is needed to ensure that the macro DLLEXPORT gets the right values.
    macros.put("mylib_DLL_BUILD", "1")

    toolChain.map { toolChain ->
        if (toolChain in Gcc) {
            compilerArgs.add("-O3")
        } else if (toolChain in Clang) {
            compilerArgs.add("-O3")
        }
    }

}

//////////////////////////////////////////////////////////////////////////////////////
// Build the JNI interface
//////////////////////////////////////////////////////////////////////////////////////

dependencies {

    // we need JavaCpp to compile
    compileOnly "org.bytedeco:javacpp:${javaCppVersion}"

    // OS dependent JavaCpp dependencies (we are using the Pointer for handling native memory)
    platformClassifier.each { platform ->
        testImplementation "org.bytedeco:javacpp:${javaCppVersion}"
        testRuntimeOnly "org.bytedeco:javacpp:${javaCppVersion}:" + platform
    }

}

// Java CPP specific settings

ext {
    // for windows we add the mingw suffix to use the MinGW-64 provided gcc compiler
    platformClassifier.each { platform ->
        javacppPlatform = (platform == "windows-x86_64" ? "windows-x86_64-mingw" : platform);
    }
}

tasks.withType(org.bytedeco.gradle.javacpp.BuildTask) {
    includePath = ["$projectDir/src/main/include"]
    linkPath = ["$projectDir/build/lib/main/release"]
}

javacppBuildCommand {
    // building of the native part of mylib is done in the above tasks

    // Hence, it is left empty intentionally.
}

javacppBuildParser {
    // this is the java classes which define the JNI interface generation
    classOrPackageNames = ['com.tnatschl.mylib.presets.*']

}

javacppBuildCompiler {
    // is set in all examples to true
    copyLibs = true
}


test {
    // we need to add the javacppJar which contains the native library to the test class path too
    classpath = project.files("$buildDir/libs/" + javacppJar.archiveName, classpath)
}

sourcesJar {
    duplicatesStrategy = DuplicatesStrategy.INCLUDE
}

Samuel Audet

unread,
Aug 3, 2021, 10:55:58 PM8/3/21
to javacpp...@googlegroups.com, Thomas Natschläger
If you're trying to use MinGW that's not a "platform". Leave javacppPlatform to "windows-x86_64", but set "properties" to "windows-x86_64-mingw" inside a block like
tasks.withType(org.bytedeco.gradle.javacpp.BuildTask) { }.

The warnings have been fixed with Gradle JavaCPP 1.5.6, so please upgrade to get rid of them.

If dependent libraries are missing from the @Platform(preload=...) annotation value, they need to be made available some other way, so if you don't want to have to add them manually everywhere, make sure that they are all specified in the @Platform(preload=...) annotation and that they actually do get copied.

Samuel

Thomas Natschläger

unread,
Aug 16, 2021, 11:02:57 AM8/16/21
to Samuel Audet, javacpp...@googlegroups.com
Dear Samuel!

Thanks for your hints. So far, everything works nicely with javaCpp 1.5.6.

-Thomas

Reply all
Reply to author
Forward
0 new messages