Cucumber-JVM: with JUnit vs. without JUnit

Robert

unread,

Oct 23, 2011, 1:27:03 PM10/23/11

to cu...@googlegroups.com

I'm not too sure if this is an issue, but I thought I'd post my findings and see if anyone else is experiencing the same behavior. Recently, I began experimenting running my tests using the --format pretty argument and without JUnit. I notice that when I run the test using JUnit, the last step, which is designed to intentionally fail, does what is expected of it (ie, fails), and Maven reports a build failure. However, when I opt to not use JUnit and instead use the --format pretty argument in conjunction with the --glue argument, the step once again fails (as expected), but Maven reports a successful build. I would have thought that the build should fail as it did when run with JUnit -- I think I might be overlooking something here :(. I've included my step def file below, along with a copy of my pom file, and the output from each run.

Test Run Results with JUnit

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.stag.hello.HelloTest

Before step

Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.538 sec <<< FAILURE!

Results :

Failed tests:

Then the expected behavior is displayed: This step is designed to intentionally FAIL

Tests run: 3, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

Test Run Results without JUnit

[INFO] --- exec-maven-plugin:1.2.1:java (default) @ simple-cucumber-jvm ---

Feature: My hello feature

Before step

Background: My background # /hello.feature:2

Given a background # HelloStepDefs.background()

Scenario: Saying hello # /hello.feature:5

Given a precondition # HelloStepDefs.givenStep()

When an action takes place # HelloStepDefs.whenStep()

Then the expected behavior is displayed # HelloStepDefs.thenStep()

junit.framework.AssertionFailedError: This step is designed to intentionally FAIL

at junit.framework.Assert.fail(Assert.java:47)

at junit.framework.Assert.assertTrue(Assert.java:20)

at org.stag.hello.HelloStepDefs.thenStep(HelloStepDefs.java:33)

at ✽.Then the expected behavior is displayed(/hello.feature:8)

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

Step Def File

package org.stag.hello;

import cucumber.annotation.Before;

import cucumber.annotation.en.Given;

import cucumber.annotation.en.Then;

import cucumber.annotation.en.When;

import static junit.framework.Assert.*;

public class HelloStepDefs {

@Before

public void beforeStep() {

System.out.println("Before step");

}

@Given("^a background$")

public void background() {

assertTrue(true);

}

@Given("^a precondition$")

public void givenStep() {

assertTrue(true);

}

@When("^an action takes place$")

public void whenStep() {

assertTrue(true);

}

@Then("^the expected behavior is displayed$")

public void thenStep() {

assertTrue("This step is designed to intentionally FAIL", false);

}

POM File (setup to run test without JUnit)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<groupId>org.stag.simplecucumber</groupId>

<artifactId>simple-cucumber-jvm</artifactId>

<version>0.0.1-SNAPSHOT</version>

<cucumber-jvm.version>1.0.0-SNAPSHOT</cucumber-jvm.version>

<junit.version>4.8.2</junit.version>

<picocontainer.version>2.13.6</picocontainer.version>

</properties>

<groupId>info.cukes</groupId>

<artifactId>cucumber-picocontainer</artifactId>

<version>${cucumber-jvm.version}</version>

</dependency>

<groupId>info.cukes</groupId>

<artifactId>cucumber-junit</artifactId>

<version>${cucumber-jvm.version}</version>

</dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>${junit.version}</version>

</dependency>

<groupId>org.picocontainer</groupId>

<artifactId>picocontainer</artifactId>

<version>${picocontainer.version}</version>

</dependency>

</dependencies>

<build>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-plugin</artifactId>

</configuration>

</plugin>

<groupId>org.codehaus.mojo</groupId>

<artifactId>exec-maven-plugin</artifactId>

<goals>

</goals>

<mainClass>cucumber.cli.Main</mainClass>

<argument>--format</argument>

<argument>pretty</argument>

<argument>org.stag.hello</argument>

</arguments>

</configuration>

</execution>

</executions>

</plugin>

</plugins>

</build>

</project>

Aslak Hellesøy

unread,

Oct 23, 2011, 1:57:14 PM10/23/11

to cu...@googlegroups.com

On Oct 23, 2011, at 19:27, Robert <rest...@gmail.com> wrote:

I'm not too sure if this is an issue, but I thought I'd post my findings and see if anyone else is experiencing the same behavior. Recently, I began experimenting running my tests using the --format pretty argument and without JUnit. I notice that when I run the test using JUnit, the last step, which is designed to intentionally fail, does what is expected of it (ie, fails), and Maven reports a build failure. However, when I opt to not use JUnit and instead use the --format pretty argument in conjunction with the --glue argument, the step once again fails (as expected), but Maven reports a successful build. I would have thought that the build should fail as it did when run with JUnit -- I think I might be overlooking something here :(.

The only thing you're doing wrong is to assume that you're doing something wrong rather than assuming that Cucumber-JVM has some bugs/incomplete features here and there.

I have created a ticket here:

https://github.com/cucumber/cucumber-jvm/issues/71

Aslak

--
You received this message because you are subscribed to the Google Groups "Cukes" group.
To view this discussion on the web visit https://groups.google.com/d/msg/cukes/-/3zu9Qz551NEJ.
To post to this group, send email to cu...@googlegroups.com.
To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/cukes?hl=en.

Robert

unread,

Oct 23, 2011, 2:03:59 PM10/23/11

to cu...@googlegroups.com

On Sunday, October 23, 2011 10:57:14 AM UTC-7, Aslak Hellesøy wrote:

On Oct 23, 2011, at 19:27, Robert <rest...@gmail.com> wrote:

I'm not too sure if this is an issue, but I thought I'd post my findings and see if anyone else is experiencing the same behavior. Recently, I began experimenting running my tests using the --format pretty argument and without JUnit. I notice that when I run the test using JUnit, the last step, which is designed to intentionally fail, does what is expected of it (ie, fails), and Maven reports a build failure. However, when I opt to not use JUnit and instead use the --format pretty argument in conjunction with the --glue argument, the step once again fails (as expected), but Maven reports a successful build. I would have thought that the build should fail as it did when run with JUnit -- I think I might be overlooking something here :(.

The only thing you're doing wrong is to assume that you're doing something wrong rather than assuming that Cucumber-JVM has some bugs/incomplete features here and there.

I have created a ticket here:
https://github.com/cucumber/cucumber-jvm/issues/71

Aslak

Thanks, Aslak.

Reply all

Reply to author

Forward