PIT command line does not recognize classes in JAR in class path

[Kevin Rosendahl]

Hi,

I'm able to get the PIT command line working if I supply the tests and class I want to mutate via a directory in the classpath, but if I put them in a jar and include that in the classpath, it stops finding the tests.

I've created a minimal reproduction:

.

├── lib

│   └── calc.jar

├── src

│   ├── Calculator.class

│   ├── Calculator.java

│   ├── CalculatorTest.class

│   └── CalculatorTest.java

└── third-party

    ├── hamcrest-core-1.3.jar

    ├── junit-4.12.jar

    ├── pitest-1.4.9.jar

    ├── pitest-command-line-1.4.9.jar

    └── pitest-entry-1.4.9.jar

Calculator.java (via junit tutorial):

public class Calculator {

  public int evaluate(String expression) {

    int sum = 0;

    for (String summand: expression.split("\\+"))

      sum += Integer.valueOf(summand);

    return sum;

  }

}

CalculatorTest.java:

import static org.junit.Assert.assertEquals;

import org.junit.Test;

public class CalculatorTest {

  @Test

  public void evaluatesExpression() {

    Calculator calculator = new Calculator();

    int sum = calculator.evaluate("1+2+3");

    assertEquals(6, sum);

  }

}

Then running with src included in the class path works:

$ java -cp src:third-party/hamcrest-core-1.3.jar:third-party/junit-4.12.jar:third-party/pitest-1.4.9.jar:third-party/pitest-command-line-1.4.9.jar:third-party/pitest-entry-1.4.9.jar \

  org.pitest.mutationtest.commandline.MutationCoverageReport \

  --reportDir reports \

  --targetClasses 'Calculator' \

  --sourceDirs src \

  --targetTests '*'

1:11:49 AM PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.

1:11:50 AM PIT >> INFO : Sending 2 test classes to minion

1:11:50 AM PIT >> INFO : Sent tests to minion

1:11:50 AM PIT >> INFO : MINION : 1:11:50 AM PIT >> INFO : Checking environment

1:11:50 AM PIT >> INFO : MINION : 1:11:50 AM PIT >> INFO : Found  1 tests

1:11:50 AM PIT >> INFO : MINION : 1:11:50 AM PIT >> INFO : Dependency analysis reduced number of potential tests by 0

1:11:50 AM PIT >> INFO : MINION : 1:11:50 AM PIT >> INFO : 1 tests received

# snip

>> Generated 2 mutations Killed 2 (100%)

>> Ran 2 tests (1 tests per mutation)

But then if I put the Calculator classes in a jar (lib/calc.jar, produced by jar --create --file lib/calc.jar src/Calculator.class src/CalculatorTest.class), PIT stops recognizing the tests:

$ java -cp lib/calc.jar:third-party/hamcrest-core-1.3.jar:third-party/junit-4.12.jar:third-party/pitest-1.4.9.jar:third-party/pitest-command-line-1.4.9.jar:third-party/pitest-entry-1.4.9.jar \

  org.pitest.mutationtest.commandline.MutationCoverageReport \

  --reportDir reports \

  --targetClasses 'Calculator' \

  --sourceDirs src \

  --targetTests '*'

1:27:03 AM PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.

1:27:04 AM PIT >> INFO : Sending 0 test classes to minion

1:27:04 AM PIT >> INFO : Sent tests to minion

1:27:04 AM PIT >> INFO : Calculated coverage in 0 seconds.

1:27:04 AM PIT >> INFO : Created  0 mutation test units

Exception in thread "main" org.pitest.help.PitHelpError: No mutations found. This probably means there is an issue with either the supplied classpath or filters.

See http://pitest.org for more details.

        at org.pitest.mutationtest.tooling.MutationCoverage.checkMutationsFound(MutationCoverage.java:287)

        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:140)

        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:120)

        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:50)

        at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:87)

        at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)

I've confirmed this behavior on AdoptOpenJDK 8 as well as OpenJDK 11.

Any thoughts on what's going wrong? Thanks!

[Dalila Amara]

This link solves the same problem in my case, 

https://stackoverflow.com/questions/32345833/pttest-is-failing-and-not-generating-mutation-coverage 

 

--
You received this message because you are subscribed to the Google Groups "PIT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pitusers+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pitusers/d2ed439e-e0d6-4485-8473-cc2cc3e54b5c%40googlegroups.com.

[Kevin Rosendahl]

Thanks for the response. Unfortunately making the targetClasses regex more permissive doesn't seem to help:

$ java -cp lib/calc.jar:third-party/hamcrest-core-1.3.jar:third-party/junit-4.12.jar:third-party/pitest-1.4.9.jar:third-party/pitest-command-line-1.4.9.jar:third-party/pitest-entry-1.4.9.jar \

org.pitest.mutationtest.commandline.MutationCoverageReport \

--reportDir reports \

--targetClasses 'Calc*' \

--sourceDirs src \

--targetTests '*'

10:30:58 AM PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.

10:30:58 AM PIT >> INFO : Sending 0 test classes to minion

10:30:58 AM PIT >> INFO : Sent tests to minion

10:30:58 AM PIT >> INFO : Calculated coverage in 0 seconds.

10:30:58 AM PIT >> INFO : Created  0 mutation test units

Exception in thread "main" org.pitest.help.PitHelpError: No mutations found. This probably means there is an issue with either the supplied classpath or filters.

See http://pitest.org for more details.

        at org.pitest.mutationtest.tooling.MutationCoverage.checkMutationsFound(MutationCoverage.java:287)

        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:140)

        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:120)

        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:50)

        at org.pitest.mutationtest.commandline.MutationCoverageReport.runReport(MutationCoverageReport.java:87)

        at org.pitest.mutationtest.commandline.MutationCoverageReport.main(MutationCoverageReport.java:45)

To unsubscribe from this group and stop receiving emails from it, send an email to pitu...@googlegroups.com.

[Dalila Amara]

This is my pom.xml after resolving the same problem, hope it will help you

To unsubscribe from this group and stop receiving emails from it, send an email to pitusers+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pitusers/d47c6413-f818-4818-9ff3-7e1b8809ceb4%40googlegroups.com.

[Kevin Rosendahl]

Thanks. Unfortunately I cannot use the maven plugin as I'm actually attempting to write a plugin for Bazel, which in turn invokes the command line with the jars it builds.

To view this discussion on the web visit https://groups.google.com/d/msgid/pitusers/d47c6413-f818-4818-9ff3-7e1b8809ceb4%40googlegroups.com.

[Henry Coles]

I think the issue you are hitting is that by default the command line tool will not mutate classes within jars. There's a flag to change this - I forget what it is (and i'm traveling just now with sketchy internet).

If you're implementing a bazal plugin i'd suggest not adding the jars to the clasdpath. Early versions of pitest did this (and the command line tool still supports it), but mixing the tool and sut clasdpath is a bad idea.

There are flags to pass jars to the tool without adding them to the clasdpath (again I forget what they are).

Looking at how the grade or maven plugin are implemented might be a good idea. I'd recommend not implementing in terms of the command line tool, its use is discouraged.

Henry

To unsubscribe from this group and stop receiving emails from it, send an email to pitusers+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pitusers/e0ec7b8a-2a57-4a5e-97f0-d80bbdf0cc01%40googlegroups.com.

[Kevin Rosendahl]

Hi Henry,

Thanks for the response while you're traveling. I've taken a look at the maven and gradle plugin implementations. It looks like gradle itself invokes the commandline in a similar way to what I was planning with Bazel: https://github.com/szpak/gradle-pitest-plugin/blob/bbe2bfe36c9a512a27de89a7551b63e5406cf113/src/main/groovy/info/solidsoft/gradle/pitest/PitestTask.groovy#L201-L209

Bazel doesn't have a Java plugin library similar to Maven's so I believe I'm stuck invoking something through the command line.

I think that the flag you're referring to may be includeJarFiles which I do see referenced as an option in the gradle plugin (https://github.com/szpak/gradle-pitest-plugin/blob/bbe2bfe36c9a512a27de89a7551b63e5406cf113/src/main/groovy/info/solidsoft/gradle/pitest/PitestTask.groovy#L228), but seems to have been removed from PIT many years ago (https://github.com/hcoles/pitest/commit/362facb13532901bf735c445f083f72e23be0d53).

It seems like the best path forward to try to get something working would be to extract the jars containing the tests and classes to mutate into a temporary directory and include that in the classpath?

Kevin

To view this discussion on the web visit https://groups.google.com/d/msgid/pitusers/e0ec7b8a-2a57-4a5e-97f0-d80bbdf0cc01%40googlegroups.com.

[Dan Zhao]

Hi Kelvin, I recently encountered the same issue, have you fixed this issue? Could you please share the solution? Thank you!