Cucumber JUnit cannot generate the correct xml result format when adding option --step-notifications

[Tam Vo]

I cloned and ran cucumber-example from https://github.com/cucumber/cucumber-jvm/tree/master/examples/java-gradle.
I configure javaexec in gradle task below:

task cucumber(dependsOn: testClasses) {
    doLast {
        javaexec {
            main = "cucumber.api.cli.Main"
            classpath = configurations.testRuntimeClasspath + sourceSets.main.output + sourceSets.test.output
            args = ['--plugin', 'pretty', '--glue', 'gradle.cucumber', 'src/test/resources', '--junit,--step-notifications', 
            '--plugin', 'junit:output.xml'
            ]
        }
    }
}

I want to Scenario or When and When, Given, Then are the test step and test case tag but in output.xml file, When is not a testcase tag now.

My Expected behavior:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="Just a failing scenario" time="0.007" tests="3" errors="2" skipped="0" failures="0">
  <testcase name="Just a failing scenario" classname="Just a failing scenario" time="0"/>
  <testcase name="I run a failing step" classname="Just a failing scenario" time="0.007">
    <error message="production failed here" type="java.lang.RuntimeException">java.lang.RuntimeException: production failed here
	at gradle.cucumber.Production.doWork(Production.java:6)
	at gradle.cucumber.BasicStepdefs.I_run_a_failing_step(BasicStepdefs.java:9)
	at ✽.I run a failing step(gradle/cucumber/gradle.feature:4)
</error>
    <system-out><![CDATA[    &amp#27;[31mWhen &amp#27;[0m&amp#27;[31mI run a failing step&amp#27;[0m       &amp#27;[90m# BasicStepdefs.I_run_a_failing_step()&amp#27;[0m
      &amp#27;[31mjava.lang.RuntimeException: production failed here
	at gradle.cucumber.Production.doWork(Production.java:6)
	at gradle.cucumber.BasicStepdefs.I_run_a_failing_step(BasicStepdefs.java:9)
	at ✽.I run a failing step(gradle/cucumber/gradle.feature:4)
&amp#27;[0m
]]></system-out>
  </testcase>
  <testcase name="Just a failing scenario" classname="Just a failing scenario" time="0">
    <error message="production failed here" type="java.lang.RuntimeException">java.lang.RuntimeException: production failed here
	at gradle.cucumber.Production.doWork(Production.java:6)
	at gradle.cucumber.BasicStepdefs.I_run_a_failing_step(BasicStepdefs.java:9)
	at ✽.I run a failing step(gradle/cucumber/gradle.feature:4)
</error>
  </testcase>
</testsuite>

And actual behavior now:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<testsuite failures="1" name="cucumber.runtime.formatter.JUnitFormatter" skipped="0" tests="1" time="0.005819">
<testcase classname="Gradle-Cucumber integration" name="Just a failing scenario" time="0.005819">
<failure message="java.lang.RuntimeException: production failed here&#10;&#9;at gradle.cucumber.Production.doWork(Production.java:6)&#10;&#9;at gradle.cucumber.BasicStepdefs.I_run_a_failing_step(BasicStepdefs.java:9)&#10;&#9;at ✽.I run a failing step(src/test/resources/gradle/cucumber/gradle.feature:4)&#10;"><![CDATA[When I run a failing step...................................................failed

StackTrace:
java.lang.RuntimeException: production failed here
	at gradle.cucumber.Production.doWork(Production.java:6)
	at gradle.cucumber.BasicStepdefs.I_run_a_failing_step(BasicStepdefs.java:9)
	at ✽.I run a failing step(src/test/resources/gradle/cucumber/gradle.feature:4)
]]></failure>
</testcase>
</testsuite>

[MP Korstanje]

Hey Tam,

while named in a similar fashion the JUnit plugin and the JUnit step notification flag are not related.

The step notification flag will map test steps to JUnit tests cases. This allows JUnit to represent and

report steps as test cases. This mapping however is fundamentally wrong and breaks quite a few

assumptions in JUnit4. As such we stopped doing this. The flag only exists for backwards compatibility.

The JUnit plugin observes events directly from Cucumber and outputs a report in the junit xml format.

You can create your own plugin to create custom reports. Have a look at the existing implementation:

https://github.com/cucumber/cucumber-jvm/blob/master/core/src/main/java/cucumber/runtime/formatter/JUnitFormatter.java

Once you've created a plugin you can run it using --plugin com.example.my.Plugin:path/to/out.xml


Cheers,
M.P.