Problems using Spock and TestNG together

[Howard Lewis Ship]

I have a challenge ... a backlog of hundreds of TestNG tests, but I'm moving towards using Spock (http://spockframework.org).

Spock is a Groovy testing environment; Spock specifications are JUnit tests; they the @RunWith annotation to hook into a custom JUnit Runner class.

I want to be able to keep most existing tests, and use Spock/JUnit in parallel. TestNG has an option for that: junit="true" on the <test> element.

However, I can't get this to work in either Gradle command line build (1.0-milestone-3), or from IDEA.

I'm using org.testng:testng:6.5.2, which itself depends on junit:junit:4.10.

I'm also using org.spockframework:spock-core:0.6-groovy-1.8  which normally depends on junit:junit-dep:4.9 (but after initial experimentation, I've turned off that transitive dependency, so that both TestNG and Spock depend on the same junit:4.10).

I've added the following to my testng.xml:

    <test name="Spock Specs" junit="true">

        <packages>

            <package name="ioc.specs"/>

        </packages>

    </test>

This ioc.specs package is new and separate from the location of all the existing TestNG tests (which are under org.apache.tapestry5.ioc, and sub-packages).

The Gradle build recognizes my new <test> but doesn't seem to recognize the Spock specifications inside.  It shows a test with 0 test cases (the TestNG tests specified by other <test> elements do execute normally).

Tweaking testng.xml to run with high verbosity, I see this when running inside IDEA:

[PackageUtils] Looking for test classes in the directory: /Users/hlship/workspaces/tapestry/tapestry5/out/test/tapestry-ioc/ioc/specs

[PackageUtils] Found class ExampleSpec, seeing it if it's included or excluded

[PackageUtils] ... Including class ExampleSpec

[TestRunner] Running the tests in 'Spock Specs' with parallel mode:false

[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@3dbbd23f priority: 10

[TestNG] Running:

  /Users/hlship/workspaces/tapestry/tapestry5/tapestry-ioc/src/test/conf/testng.xml

[SuiteRunner] Created 1 TestRunners

[TestRunner] Running test Spock Specs on 0  classes,  included groups:[] excluded groups:[]

[JUnit4TestMethod] Method 'sky_is_not_blue' not found in class 'ioc.specs.ExampleSpec': ioc.specs.ExampleSpec.sky_is_not_blue()

===== Invoked methods

=====

Has anyone else managed to get Spock and TestNG to co-exist. I'm pretty reliant on running my tests from the IDE, and I vastly prefer to have one IDE command to run the entire gamut of TestNG and Spock/JUnit tests in one go, rather than separate commands for each (I believe I could get Gradle to run TestNG and JUnit tests separately, though I expect some issues related to report generation if I go down that path).

--
Howard M. Lewis Ship

Creator of Apache Tapestry

The source for Tapestry training, mentoring and support. Contact me to learn how I can get you up and productive in Tapestry fast!

(971) 678-5210
http://howardlewisship.com

[Peter Niederwieser]

To my knowledge, TestNG can only run JUnit3 tests, but not JUnit4 tests. In theory, execution environments like IDEs and build tools could support running TestNG and JUnit4 tests in one go, but I don't know of any that do. In Maven you need two Surefire executions. In Gradle you need two test tasks (and will get two reports). Not sure about IDEs.

If you find a way, let us know. But I'm not optimistic.

Cheers,

Peter

--
You received this message because you are subscribed to the Google Groups "Spock Framework - User" group.
To post to this group, send email to spockfr...@googlegroups.com.
To unsubscribe from this group, send email to spockframewor...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/spockframework?hl=en.

[Peter Nedonosko]

Hello,

I confirm a problem with Spock and TestNG together in single Maven 3.0.5 project. I'm adopting Spock for my project and after all obvious (and documented) steps I can run the *Spec.class tests... but with a small notice from initial message in this post: they do compile and run (I see what I print in setup()) but nothing executed for the spec methods itself. Finally I see:

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Final though came from an attempt to use Test annotation to point for a test method explicitly in code and there I found that I have two such Test classes on the classpath. By excluding the dependency of TestNG, the methods from the spec start to execute. But I still need my (old) TestNG tests.

Is there an example Surefire setup how to let both, Spock and TestNG, to co-exist in a single project? 

[Marcin Zajączkowski]

On 2014-02-11 22:26, Peter Nedonosko wrote:
> Hello,
>
> I confirm a problem with Spock and TestNG together in single Maven 3.0.5
> project. I'm adopting Spock for my project and after all obvious (and
> documented) steps I can run the *Spec.class tests... but with a small
> notice from initial message in this post: they do compile and run (I see
> what I print in setup()) but nothing executed for the spec methods itself.
> Finally I see:
>
> Results :
>
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Hi Peter,

I haven't tried a configuration with TestNG running JUnit/Spock tests,
bt as a workaround for the transition period you could configure Maven
(surefire/failsafe) to run both JUnit/Spock and TestNG tests. See ma
blog post:
http://solidsoft.wordpress.com/2013/03/12/mixing-testng-and-junit-tests-in-one-maven-module-2013-edition/

It is just for TestNG and JUnit (without Spock), but as Spock was
written as JUnit runner it generally should just work (despite the fact
you have to configure surefire/failsafe to find tests in your *Spec
files). See sample project by Masahiko Sakamoto:
https://github.com/msakamoto-sf/maven3-junit-spock-testng-mixin

Marcin

>> To post to this group, send email to spockfr...@googlegroups.com<javascript:>
>> .

>> To unsubscribe from this group, send email to

>> spockframewor...@googlegroups.com <javascript:>.

>> For more options, visit this group at
>> http://groups.google.com/group/spockframework?hl=en.
>>
>>
>>
>

--
http://blog.solidsoft.info/ - Working code is not enough

[Peter Nedonosko]

Thank you Marcin for a great suggestion in your blog!
I found your article by googling and built my pom.xml from what Masahiko compiled in his github. Together it helped me to run new Spock specs and keep legacy made with TestNG. 

Kudos!