JUnit5 / Java8 adoption (#141)

[Paul Hicks]

I've been experimenting with gradle modules, sourceSets and whatnot in order to get java6 and java8 builds working nicely together. I have a passable implementation, which means I'm able to move on to JUnit5. Is JUnit5 work underway, and if so, can I help?

To get up to speed, I've been trying to design a solution for JUnit5 that uses a TestEngine for Concordion, since that fits best with JUnit's ideas of engines versus extensions: use different engines for running different types of tests/specs, and use extensions for smaller tweaks to the way things run. In trying to stick close to the existing JUnit4 implementation, I've not yet been able to sufficiently distinguish between the Jupiter engine and a potential Concordion engine to make it work. So I've redesigned a little and ended up with a choice which I'd like input on.

1. Stick with the existing JUnit4+Concordion design, drop the idea of using a ConcordionTestEngine, and switch exclusively to the JUnit5 extension API. This will be easy, won't take long to implement, and can borrow a lot of code from the existing JUnit4 integration.
2. Forget the JUnit4 way of doing things, create a new ConcordionTestEngine that discovers specs first, and then discovers fixtures from the specs. This would put (slightly) more constraints on the locations and naming of spec files, but would eliminate @Test annotations from fixtures. It might be a little scary at first but would mean no JUnit code in the fixtures (well, there would be assumptions and assertions, but you wouldn't need JUnit for that, you can go with AssertJ, Hamcrest, Fest, etc..). This will take longer and wouldn't be able to leverage much of the existing code.

[Tim Wright]

Hi Paul,

I've done some very rudimentary investigation into JUnit5 integration and have some vague thoughts - basically I was contemplating using the @TestFactory annotation. It returns a lazily evaluated stream of tests. If it's properly lazily evaluated, then we should be able to add new fixtures discovered from concordion:run (and examples from the fixtures) to the stream and have them executed.

But I haven't actually written code to test that hypothesis yet. And, to be honest, I'm time poor at the moment.

Tim

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

--
You received this message because you are subscribed to the Google Groups "concordion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concordion-dev+unsubscribe@googlegroups.com.
To post to this group, send email to concordion-dev@googlegroups.com.
Visit this group at https://groups.google.com/group/concordion-dev.
To view this discussion on the web, visit https://groups.google.com/d/msgid/concordion-dev/c85b506d-b41f-465c-ae1d-d8ce428463f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Wright]

On saying that, I have found some time tonight. 

Here is a PR that updates our junit compile dependancy to junit 5 - it has a 'vintage' mode for junit3 and 4 emulation. Seems to work!

I'll be updating the PR as things come to mind - so don't merge yet Nigel!

Tim

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

On 4 July 2017 at 19:15, Tim Wright <t...@tfwright.co.nz> wrote:

Hi Paul,

I've done some very rudimentary investigation into JUnit5 integration and have some vague thoughts - basically I was contemplating using the @TestFactory annotation. It returns a lazily evaluated stream of tests. If it's properly lazily evaluated, then we should be able to add new fixtures discovered from concordion:run (and examples from the fixtures) to the stream and have them executed.

But I haven't actually written code to test that hypothesis yet. And, to be honest, I'm time poor at the moment.

Tim

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

On 4 July 2017 at 09:35, Paul Hicks <ten...@gmail.com> wrote:

I've been experimenting with gradle modules, sourceSets and whatnot in order to get java6 and java8 builds working nicely together. I have a passable implementation, which means I'm able to move on to JUnit5. Is JUnit5 work underway, and if so, can I help?

To get up to speed, I've been trying to design a solution for JUnit5 that uses a TestEngine for Concordion, since that fits best with JUnit's ideas of engines versus extensions: use different engines for running different types of tests/specs, and use extensions for smaller tweaks to the way things run. In trying to stick close to the existing JUnit4 implementation, I've not yet been able to sufficiently distinguish between the Jupiter engine and a potential Concordion engine to make it work. So I've redesigned a little and ended up with a choice which I'd like input on.

1. Stick with the existing JUnit4+Concordion design, drop the idea of using a ConcordionTestEngine, and switch exclusively to the JUnit5 extension API. This will be easy, won't take long to implement, and can borrow a lot of code from the existing JUnit4 integration.
2. Forget the JUnit4 way of doing things, create a new ConcordionTestEngine that discovers specs first, and then discovers fixtures from the specs. This would put (slightly) more constraints on the locations and naming of spec files, but would eliminate @Test annotations from fixtures. It might be a little scary at first but would mean no JUnit code in the fixtures (well, there would be assumptions and assertions, but you wouldn't need JUnit for that, you can go with AssertJ, Hamcrest, Fest, etc..). This will take longer and wouldn't be able to leverage much of the existing code.

--
You received this message because you are subscribed to the Google Groups "concordion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concordion-dev+unsubscribe@googlegroups.com.

To post to this group, send email to concord...@googlegroups.com.

[Tim Wright]

Helps to include the link: https://github.com/concordion/concordion/pull/243

Tim

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

[Paul Hicks]

This is exactly the solution I used for another spec-by-example tool that begins with C. And with the new DynamicNode API in JUnit5 M4, it's even easier now.

Poetically, this solution is no longer suitable in V2 of the other tool :) I'll have to go with the TestEngine solution there.

[Paul Hicks]

The JUnit5 branch won't compile with Java6: it requires Java8 because of the JUnit5 dependencies:

bad class file: ZipFileIndexFileObject[/home/paulh/.gradle/caches/modules-2/files-2.1/org.junit.jupiter/junit-jupiter-api/5.0.0-M4/50f95d68c674a30374c5f50e6ad5212d2c1a3cec/junit-jupiter-api-5.0.0-M4.jar(org/junit/jupiter/api/Test.class)]
class file has wrong version 52.0, should be 50.0
Please remove or make sure it appears in the correct subdirectory of the classpath.
import org.junit.jupiter.api.Test;

Is this important? If it is, I have a fix for this that I did a few days again in GitHub. If you like I can rebase my work onto your branch and submit a PR to your repo?

[Nigel Charman]

I'll be interested to see pros and cons of TestEngine vs @TestFactory.

For example, if I wanted to run Concordion Spring tests, would I be able
to mix the Spring and Concordion runners? What else do we need to consider?

[Paul Hicks]

The TestEngine solution would completely replace the Jupiter engine, exactly like Jupiter replaces Vintage. You can't use @RunWith in Jupiter, and you wouldn't be able to use @ExtendWIth with a ConcordionTestEngine, unless the engine was written to support that. Probably by extending the existing JupiterTestEngine.

Wanting to use the JUnit-provided SpringExtension is an excellent argument against the TestEngine approach. The @TestFactory would definitely work. A more general solution along the lines of @ExtendWith(CucumberExtension.class) providing a ParameterResovler for the Concordion class is also worth considering. This may even be an opportunity to go one further and provide classes that map to DynamicContainer and DynamicTest.

The soon-to-be-released milestone 5 of JUnit5 makes this sort of stuff easier. They've just closed the milestone (11 hours ago), so expect to see M5 available in jcenter/mavenCentral rsn.

[Paul Hicks]

Just checked on search.maven.org. M5 is out now. See [the JUnit5 user guide](http://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests) for info about DynamicContainer. I have some sample code from last month's Kotlin presentation that uses it, it's very nice.

[Paul Hicks]

Heh. The DynamicContainer stuff is very nice... my code is decidedly average :)

[Tim Wright]

Thanks for the code Paul!

I've made good progress with the junit5 integration. Basically, I've got tests running (and passing) that check that the @BeforeEach and @BeforeAll annotations are being processed correctly. The concordion concordion:example='before annotation doesn't work (yet), but that isn't far away now...

There's a lot of cleanup to do (I copied and pasted lots of code out of the ConcordionRunner to get things working).

The user interface is a bit ugly at the moment. Basically, I've got the minimum code a tester has to write down to this. I'd love to reduce it further...hopefully reading about DynamicContainer might mean we can get away without having a @TestFactory method - without this method, junit won't actually run the tests at the moment.

@ConcordionJUnit5
public class Junit5Test {
    @ConcordionTests
    public Iterable<DynamicTest> concordionTests;

    @TestFactory
    Iterable<DynamicTest> determineConcordionTests() {
        return concordionTests;
    }
}

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

On 5 July 2017 at 16:43, Paul Hicks <ten...@gmail.com> wrote:

Heh. The DynamicContainer stuff is very nice... my code is decidedly average :)

--

You received this message because you are subscribed to the Google Groups "concordion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concordion-dev+unsubscribe@googlegroups.com.
To post to this group, send email to concordion-dev@googlegroups.com.
Visit this group at https://groups.google.com/group/concordion-dev.

To view this discussion on the web, visit https://groups.google.com/d/msgid/concordion-dev/f98ded53-cbb9-4d1a-af98-e7ec71686431%40googlegroups.com.

[Tim Wright]

Actually, found a shorter way - make our extension a ParameterResolver as well so we can inject the tests into our @TestFactory method.:

@ConcordionJUnit5
public class Junit5ParamResolverTest {
    @TestFactory
    Iterable<DynamicTest> determineConcordionTests(Iterable<DynamicTest> concordionTests) {
        return concordionTests;
    }
}

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

On 5 July 2017 at 20:46, Tim Wright <t...@tfwright.co.nz> wrote:

Thanks for the code Paul!

I've made good progress with the junit5 integration. Basically, I've got tests running (and passing) that check that the @BeforeEach and @BeforeAll annotations are being processed correctly. The concordion concordion:example='before annotation doesn't work (yet), but that isn't far away now...

There's a lot of cleanup to do (I copied and pasted lots of code out of the ConcordionRunner to get things working).

The user interface is a bit ugly at the moment. Basically, I've got the minimum code a tester has to write down to this. I'd love to reduce it further...hopefully reading about DynamicContainer might mean we can get away without having a @TestFactory method - without this method, junit won't actually run the tests at the moment.

@ConcordionJUnit5
public class Junit5Test {
    @ConcordionTests
    public Iterable<DynamicTest> concordionTests;

    @TestFactory
    Iterable<DynamicTest> determineConcordionTests() {
        return concordionTests;
    }
}

--

Tim

021 251 5593

http://www.linkedin.com/in/drtimwright 

On 5 July 2017 at 16:43, Paul Hicks <ten...@gmail.com> wrote:

Heh. The DynamicContainer stuff is very nice... my code is decidedly average :)

--
You received this message because you are subscribed to the Google Groups "concordion-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concordion-dev+unsubscribe@googlegroups.com.

To post to this group, send email to concord...@googlegroups.com.

Visit this group at https://groups.google.com/group/concordion-dev.