Jacoco xml report package does not include sub project name

[Michelle]

Hello,

Jacoco html reports works for me, but for xml reports, the package name does not include the sub project name. Could you please let me know how to add that. my gradle.build file is:

allprojects {
    apply plugin: 'jacoco'

    repositories {
        jcenter()
    }

    jacoco {
        toolVersion = '0.8.1'
    }
}
subprojects {
    apply plugin: 'kotlin'
    apply plugin: "kotlin-spring"
    apply plugin: 'io.spring.dependency-management'

    repositories {
        mavenCentral()
        maven { url 'https://repo.spring.io/libs-snapshot' }
        maven { url 'https://repo.spring.io/libs-milestone' }
    }

    dependencyManagement {
        imports {
            mavenBom "org.springframework.boot:spring-boot-dependencies:${spring_boot_version}"
            mavenBom 'org.springframework.cloud:spring-cloud-dependencies:Finchley.M9'
        }
    }

    dependencies {
        compile "com.github.wnameless:json-flattener:0.5.0"
        compile "com.google.guava:guava:$guava_version"
        compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
        compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"

        testCompile "junit:junit:$junit_version"
        testCompile "org.mockito:mockito-core:$mockito_version"
    }

    compileKotlin {
        kotlinOptions.jvmTarget = "1.8"
    }
    compileTestKotlin {
        kotlinOptions.jvmTarget = "1.8"
    }

    jacocoTestReport {
        sourceDirectories = files(sourceSets.getByName("main").allSource.srcDirs.findAll{ dir -> dir.toString().contains("kotlin")})
        classDirectories =  files(sourceSets.getByName("main").output.findAll{ dir -> dir.toString().contains("kotlin") })
        println("**********source files******")
        println(sourceDirectories.files.toListString())
        println(classDirectories.files.toListString())
        println("********class files end")


        reports {
            html.enabled = true
            xml.enabled = true
            csv.enabled = false
        }
    }
}
//noinspection GroovyAssignabilityCheck
task jacoco(type:JacocoReport) {
    dependsOn subprojects.collect { it.tasks.getByPath('check')}
    sourceDirectories = files()
    classDirectories = files()
    executionData = files()

    getSubprojects()
        .findAll{ subproject ->
            subproject.name == "service-activity" }
        .forEach{
        subproject ->
            def coverageFileLocation = "${subproject.buildDir}/jacoco/test.exec"
            if (new File(coverageFileLocation).exists()) {
                executionData += subproject.tasks.findByName("jacocoTestReport").property("executionData")
                sourceDirectories += subproject.tasks.findByName("jacocoTestReport").property("sourceDirectories")
                classDirectories += subproject.tasks.findByName("jacocoTestReport").property("classDirectories")
            }
        }
    reports {
        xml.enabled = true
        csv.enabled = false
        html.enabled = true

    }
}

task jacocoFix(type: Copy) {
    from 'build/reports/jacoco/jacoco/jacoco.xml' into './'
    filter {
        line -> line.replaceAll("<package name=\"com/cd/activity\">", "<package name=\"service-activity/src/main/kotlin/com/cd/activity\">")
    }
    filter { line -> line.replaceAll("<sourcefile name=\"ServerResponseExtensions.kt\">", "<sourcefile name=\"ActivityWebHandlers.kt\">")
    }
    filter { line -> line.replaceAll("<sourcefile name=\"ServerRequestExtensions.kt\">", "<sourcefile name=\"ActivityWebHandlers.kt\">")
    }
    filter { line -> line.replaceAll("<sourcefile name=\"Extensions.kt\">", "<sourcefile name=\"ActivityWebHandlers.kt\">")
    }

}

task wrapper(type: Wrapper) {
    gradleVersion = '4.5.1' //version required
}

I use jacocofix to fix the package name, but I have a lot subprojects, I can not do this. also I do not know why the Extenstions kt files are there.

this is gradle.build config for multi kotlin projects.

Thank you for your response in advance.

Michelle

[Marc Hoffmann]

Hi Michelle,

the JaCoCo XML report uses the “package” tag aggregate a Java package. Therefore the name is the name of the Java package.

Packages can be nested into groups with arbitrary names. Check the parent “group” element to see whether it contains the correct information.

Other wise please contact the grade project. We’re not doing the JaCoC integration for grade ourself.

Regards,

-marc

--
You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/4ff3c538-f25a-47b2-8646-c9061bd614cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Evgeny Mandrikov]

In addition to what Marc said: "ServerResponseExtensions.kt" - is a name in one of analyzed class files produced by Kotlin compiler. In particular maybe one of analysed class directories contains JAR files of your dependencies.

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+unsubscribe@googlegroups.com.

[Michelle]

Ok, thank you for your response!. 

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.

[Michelle]

Thank you both for the reply! I will ask CodeClimate.

[Michelle]

Hey Evgeny and Marc,

I understood the "ServerRespsonseExtensions.kt" is a dependency. I tried to exclude them, but did not succeed. Could you please let me know if I can exclude the "Extensions.kt" in the sourceDirectories. If so, could you please let me know how to do it correctly.

Here is what I did, but I do not think it is correct:

def excludes = [
      '**/**ActivityWebHandlers.kt',
        '**/**ServerRequestExtensions.kt',
        '**/**ServerResponseExtensions.kt',
        ]
def sourceDir = sourceSets.getByName("main").allSource.srcDirs.find{ dir -> dir.toString().contains("kotlin")}

def sourceTree = fileTree(

        dir: sourceDir,
        excludes: excludes
)
sourceDirectories = files([sourceTree])

Thank you very much,

Michelle

On Monday, 23 July 2018 06:28:15 UTC-5, Evgeny Mandrikov wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.

[Evgeny Mandrikov]

Hi Michelle,

First of all - here is what Marc already said about Gradle:

On Monday, July 23, 2018 at 12:46:00 PM UTC+2, Marc R. Hoffmann wrote:

Other wise please contact the grade project. We’re not doing the JaCoC integration for grade ourself.

And as was said many times in this mailing list, just snippet of build.gradle or pom.xml doesn't fully demonstrate problem/difficulty and hence almost never enough to provide a good answer.

So I will try to guess based on search of "gradle jacoco exclude classes" in Google, which shows https://stackoverflow.com/questions/29887805/filter-jacoco-coverage-reports-with-gradle

From this page seems that classes should be excluded and not sources as you do.

What actually makes sense, because JaCoCo performs analysis of classes and works even in absence of source files.

Finally in attachment you'll find minimalistic example based on above link, execution of "gradle clean test jacocoTestReport" for which will generate report with exclusion of "a/A.class" that is compiled from "a/A.kt".

You need to do similarly - we can't guess them, but you need to exclude class files that are compiled from yours "ServerResponseExtensions.kt". In particular if source file contains top-level functions, then you need to exclude "ServerResponseExtensionsKt.class".

Regards,

Evgeny

[Evgeny Mandrikov]

forgotten attachment

[Michelle]

I added up with the jacocoFix like this:

task jacocoFix(type: Copy) {
    from 'build/reports/jacoco/jacocoRootReport/jacocoRootReport.xml' into './'
    rename { fileName -> "jacoco.xml"}
    filter { line -> line.replaceAll("</sourcefile><sourcefile name=\"ServerResponseExtensions.kt\">", "")
    }
    filter { line -> line.replaceAll("</sourcefile><sourcefile name=\"ServerRequestExtensions.kt\">", "")
    }
    filter { line -> line.replaceAll("</sourcefile><sourcefile name=\"Extensions.kt\">", "")
    }
}

filter out them for CodeClimate coverage reports

[Michelle]

Thank you Evgency for the reponse, the weird thing is no Extensions.class in the compiles. I should ask gradle. For now, I can filter them out, then send to code climate. Thank you though!

[Evgeny Mandrikov]

I hope you realize that you are not doing proper removal of nodes from XML and  actually damaging XML report - for example

<sourcefile name="Foo.java">

bar

</sourcefile><sourcefile name="ServerResponseExtensions.kt">

baz

</sourcefile>

after your hack will become

<sourcefile name="Foo.java">

bar

baz

</sourcefile>

what has completely different semantic.

[Michelle]

Hey Evengy,

I knew this, just think codeclimate will only read bar and ignore baz as now. Looks like the coverage report number is correct so far. 

However, I do not like this solution and like to have as possible as better one. the solution you told, I could not use it. Because class files does not include the Extensions.class. so even I do that, the The extensions.kt are still in the source files. Do you know any one have this similar issue? or should go ask Kotlin or Gradle?

Thanks,

Michelle

[Evgeny Mandrikov]

On Wednesday, August 1, 2018 at 5:16:40 PM UTC+2, Michelle wrote:

Hey Evengy,

I knew this, just think codeclimate will only read bar and ignore baz as now. Looks like the coverage report number is correct so far. 

However, I do not like this solution and like to have as possible as better one. the solution you told, I could not use it. Because class files does not include the Extensions.class. so even I do that, the The extensions.kt are still in the source files. Do you know any one have this similar issue? or should go ask Kotlin or Gradle?

As was said before: "Extensions.kt" - is a name in one of analyzed class files produced by Kotlin compiler.

Similarly to the names recorded by Java compiler:

e.g. in case of "class Foo { public static void main(String[] args) { } }" as "Example.java" in directory "src" execution of

javac src/Example.java

javap -v Foo.class | grep "Compiled from"

shows the name recorded by compiler as

  Compiled from "Example.java"

Notice that this source file name is completely different from class file name.

Exactly this name is extracted by JaCoCo from class files - for the same above example execution of

java -javaagent:jacoco-0.8.1/lib/jacocoagent.jar Foo

java -jar jacoco-0.8.1/lib/jacococli.jar report jacoco.exec --classfiles Foo.class --xml report.xml

produces XML report that contains "Example.java"

Note that commands after "javac" doesn't even know about existence of "src" directory and file "Example.java" in it - you can even remove it right after execution of "javac" and result of subsequent commands will remain the same.

And there is no other ways it can appear in XML. So check more carefully class files in directories, subdirectories and archives (JARs, etc) that you analyze.

Or provide us full reproducer instead of snippets of build.gradle.

[Michelle]

Hey Evgeny,

We know those files are created from AcitivityWebHandlers.kt, because of the serverRequest.bodyToMono<Activity>, it use library Extensions stuff. but might be a better way to do it or you have a better idea to help this.

I like to send you the project which I cut, but can do ./gradlew check and ./gradlew jacoco to create the jacoco.xml in build/reports/jacoco/jacoco/jacoco.xml

Can I sent it to your email, instead of here.

Thank you,

Michelle

[Evgeny Mandrikov]

On Wednesday, August 1, 2018 at 11:22:12 PM UTC+2, Michelle wrote:

Hey Evgeny,

We know those files are created from AcitivityWebHandlers.kt, because of the serverRequest.bodyToMono<Activity>, it use library Extensions stuff. but might be a better way to do it or you have a better idea to help this.

If you know the class files, then

as was said in https://groups.google.com/d/msg/jacoco/6O5R8fHPrIc/h1yhcunyCQAJ

you need to configure Gradle to exclude them

similarly to example attached in https://groups.google.com/d/msg/jacoco/6O5R8fHPrIc/G-ywuvnyCQAJ

[Evgeny Mandrikov]

On Thu, Aug 2, 2018 at 7:22 AM Michelle Gao <michell...@gmail.com> wrote:

Sure Evgeny, I attached the part of project, but can compile and create the jacoco.xml 

I also added: 

afterEvaluate {
        classDirectories = files(classDirectories.files.collect {
            fileTree(dir: it, exclude: ['**/**WebHandlers.class'])
        })
    }

in the jacocoTestReport, but did not see it works.

We are having multi subprojects, do not know how does it affect. Also somewhere use excludes. Do they make difference  or not.

thank you!

Michelle

As I can see from build of your projects the name of class file that references "ServerRequestExtensions.kt"

is actually "ActivityWebHandlers$create$$inlined$bodyToMono$1.class"

so of course it won't be matched by pattern "*/WebHandlers.class"

Before attached patch command

gradle clean build jacocoTestReport

produces report in "service-activity/build/reports/jacoco"

You also have task "jacoco", so after first command

gradle jacoco

produces report in "build/reports/jacoco"

After attached patch

and

respectively. And they disappear from XML also:

cat service-activity/build/reports/jacoco/test/jacocoTestReport.xml | grep "Extensions.kt" | wc -l

0

cat build/reports/jacoco/jacoco/jacoco.xml | grep "Extensions.kt" | wc -l

0

Regards,

Evgeny

[Michelle]

Hey Evgeny,

Thank you very much for spending time finding this for me. This is awesome!!! I will remove my jacoco fix code. 

Thank you again, you are awesome!

Michelle

[Michelle]

Hey Evgeny,

Instead of filter out *Webhandlers*, I filtered out *body* which filtered out all ServerRequestExtensions.kt and ServerResponseExtensions.kt. But some of my sub project also created Extensions.kt. I tried to figure out, but after download Kotlin, tried for a while, still has not figured out yet. I am not sure which is the one from the html reports classes.

I attached another part project to you, could you please run it again and let me know which class I should filter out.  At same time, I would be very appreciate if you can let me know how to do you find the compiled classes.

Thank you very much again!

Michelle

[Michelle]

Hey Evgeny,

I think I figured out. it looks like kafaka inlined.read caused. 

Thank you very much though

Michelle

[Evgeny Mandrikov]

On Thursday, August 2, 2018 at 8:28:23 PM UTC+2, Michelle wrote:

At same time, I would be very appreciate if you can let me know how to do you find the compiled classes.

Simple textual search of source file name in class files:

grep "ServerRequestExtensions.kt" -r .

And "javap" command for confirmation as was shown in https://groups.google.com/d/msg/jacoco/6O5R8fHPrIc/HjT6JESNDAAJ

[Michelle]

Thank you very much, Evgeny! they work great for me. You are absolutely  awesome!!!