Exclude unit tests from executing during PIT run

[Jim Kaib Jr.]

.d Evening,

I know this sounds counterintuitive, but I'd like to prevent some unit test classes from executing during a PIT analysis.  I've just started using PIT, first at the class level using filters, and now I'd like to evaluate the entire application using the Maven plugin.  My challenge is that I have a lot of EvoSuite (v1.0.5) test classes that run alongside hand-written tests, sometimes against the same production class.  While a lot of classes are mutated/tested correctly, I see a lot of errors like the one below in the Maven build log.  As far as I can tell, an EvoSuite test class (my suffix: EsTest) is associated with each occurence of this error. 

Since I have a lot of EvoSuite-generated test classes, with more to come, I'd like to simply exclude them during a PIT analysis if that will silence the error.  The Maven build I'm working on uses four bytecode modification tools (JMockit, JaCoCo, EvoSuite, and now PIT).  I know getting all of them to play nicely together can be tough, and since I included a mutation-based strategy when I generated the EvoSuite tests, life may be easier if could exclude them during a PIT analysis.

Of course, this assumes I triaged the problem correctly. ;)

Cheers!

stdout  : Installing PIT agent
stderr  : 6:18:01 PM PIT >> FINE : Running mutation MutationDetails [id=MutationIdentifier [location=Location [clazz=com.foo.InvoiceLineItem, method=isIso, methodDesc=()Z], indexes=[6], mutator=org.pitest.mutationtest.engine.gregor.mstderr  : utators.RemoveConditionalMutator_EQUAL_IF], filename=InvoiceLineItem.java, block=151, lineNumber=741, description=removed conditional - replaced equality check with true, testsInOrder=[com.foo.InvoiceableItemEsTest.com.fplestderr  : .fim.phoenix.intf.domain.invoice.InvoiceableItemEsTest, com.foo.BalanceSheetItemEsTest.com.foo.BalanceSheetItemEsTest, com.foo.MyOutboundAccrualEsTest.com.fstderr  : ple.fim.phoenix.intf.domain.invoice.MyOutboundAccrualEsTest, com.foo.MyOutboundActualEsTest.com.foo.MyOutboundActualEsTest], isInFinallyBlock=false, poison=NORMAL]
stderr  : 6:18:01 PM PIT >> FINE : mutating method isIso
stderr  : 6:18:01 PM PIT >> FINE : 4 relevant test for isIso
stdout  : java.lang.UnsupportedOperationException: class redefinition failed: attempted to change superclass or interfaces
stdout  :     at sun.instrument.InstrumentationImpl.redefineClasses0(Native Method)stdout  :
stdout  :     at sun.instrument.InstrumentationImpl.redefineClasses(InstrumentationImpl.java:170)stdout  :
stdout  :     at org.pitest.boot.HotSwapAgent.hotSwap(HotSwapAgent.java:46)stdout  :
stdout  :     at org.pitest.mutationtest.execute.HotSwap.apply(HotSwap.java:28)stdout  :
stdout  :     at org.pitest.mutationtest.execute.HotSwap.apply(HotSwap.java:9)stdout  :
stdout  :     at org.pitest.mutationtest.execute.MutationTestWorker.handleCoveredMutation(MutationTestWorker.java:144)
stdout  :     at org.pitest.mutationtest.execute.MutationTestWorker.handleMutation(MutationTestWorker.java:125)
stdout  :     at org.pitest.mutationtest.execute.MutationTestWorker.processMutation(MutationTestWorker.java:106)
stdout  :     at org.pitest.mutationtest.execute.MutationTestWorker.run(MutationTestWorker.java:77)
stdout  :     at org.pitest.mutationtest.execute.MutationTestMinion.run(MutationTestMinion.java:88)
stdout  :     at org.pitest.mutationtest.execute.MutationTestMinion.main(MutationTestMinion.java:119)
stderr  : 6:18:01 PM PIT >> WARNING : Error during mutation test
java.lang.UnsupportedOperationException: class redefinition failed: attempted to change superclass or interfaces
    at sun.instrument.InstrumentationImpl.redefineClasses0(Native Method)
    at sun.instrustderr  : ment.InstrumentationImpl.redefineClasses(InstrumentationImpl.java:170)
    at org.pitest.boot.HotSwapAgent.hotSwap(HotSwapAgent.java:46)
    at org.pitest.mutationtest.execute.HotSwap.apply(HotSwap.java:28)
    at org.pitest.mutationtest.execute.HotSwap.apply(Hostderr  : tSwap.java:9)
    at org.pitest.mutationtest.execute.MutationTestWorker.handleCoveredMutation(MutationTestWorker.java:144)
    at org.pitest.mutationtest.execute.MutationTestWorker.handleMutation(MutationTestWorker.java:125)
    at org.pitest.mutationtest.executstderr  : e.MutationTestWorker.processMutation(MutationTestWorker.java:106)
    at org.pitest.mutationtest.execute.MutationTestWorker.run(MutationTestWorker.java:77)
    at org.pitest.mutationtest.execute.MutationTestMinion.run(MutationTestMinion.java:88)
    at org.pitesstderr  : t.mutationtest.execute.MutationTestMinion.main(MutationTestMinion.java:119)
6:18:01 PM PIT >> FINE : Exit code was - UNKNOWN_ERROR
6:18:01 PM PIT >> WARNING : Minion exited abnormally due to RUN_ERROR
6:18:01 PM PIT >> FINE : Setting 1 unfinished runs to RUN_ERROR state

[henry]

You can exclude tests using the imperfectly named excludedClasses parameter (it excludes both matching tests from being run and classes from being mutated).

As it makes no sense to have both JaCoco and pitest active at the same time, so you should also be able to remove one of the bytecode manipulators from your pitest run.

I'm afraid I can't guess whether it is an interaction with evosuite or JMockit (or both in combination) that is causing this issue.

Henry