Generate Serenity aggregate report when tests are executed with cucumber-jvm-parallel-plugin

[Nicolai Baidan]

I've created a question on stackoverflow , but will also post here for better visibility.

https://stackoverflow.com/questions/46096442/generate-serenity-aggregate-report-when-tests-are-executed-with-cucumber-jvm-par

Since Serenity, doesn't support parallel execution out of the box ,I'm using Cucumber jvm parallel plugin , after the tests are executed successfully I get the following files in my target/failsafe-reports directory:

failsafe-summary.xml
Parallel01IT.txt
Parallel02IT.txt
Parallel03IT.txt
TEST-Parallel01IT.xml
TEST-Parallel02IT.xml
TEST-Parallel03IT.xml

After I run mvn sernity:aggregate I get this:

[INFO] Generating test results for 0 tests
[INFO] 2 requirements loaded after 80 ms
[INFO] 2 related requirements found after 80 ms
[INFO] Generating test outcome reports: false
[INFO] Starting generating reports: 92 ms
[INFO] Configured report threads: 40
[INFO] Test results for 0 tests generated in 352 ms

For some reason the report aggregator does not seem to find the file locations. If I run the tests sequentially the report works just fine , even though the results are stored in the same directory.

If also tried setting the sourceDirectory in report plugin , but to no avail.

Is there some configuration options I am missing ? Or is it just plainly not possible to generate the report if I'm using the parallel plugin ?

[John Smart]

Did you use a custom template so that the tests run with the CucumberWithSerenity runner?

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

___________________________________________________
John Smart | Wakaleo Consulting  |  +44 7398 832273
Making smart teams collaborate better
http://johnfergusonsmart.com  |  john....@wakaleo.com
___________________________________________________

We love breaking down silos and helping smart teams collaborate better! Ask about our tailored on-site workshops in Agile Product Planning, BDD Requirements Discovery,  BDD, TDD and Clean Coding, and Advanced BDD Test Automation.

Need some help with Serenity BDD? Check out our Serenity BDD training and support packages here.

___________________________________________________

[Nicolai Baidan]

@RunWith(CucumberWithSerenity.class)
@CucumberOptions(features = "src/test/resources/features/", glue = "ssf.definitionSteps")
@WebAppConfiguration
public class DefinitionTestSuite {

}

четверг, 7 сентября 2017 г., 15:50:01 UTC+3 пользователь John Smart написал:

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send email to thucydid...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[John Smart]

Serenity should have generated some JSON files in the target/site/serenity directory - can you see any of them?

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nicolai Baidan]

I've did a little bit more digging , regarding your question with the runner .

The actual runners generated by the plugin , did not use CucumberWithSerenity.

I've created a custom template based on https://github.com/temyers/cucumber-jvm-parallel-plugin/blob/master/src/main/resources/cucumber-junit-runner.java.vm and set the path to it 

with   <customVmTemplate>src/test/resources/cucumber-custom-runner.vm</customVmTemplate> 

Afterwards the report is successfully generated.

Thank you very much for your help, as always your insight is very helpful and on point.

I will make sure to update the the stackoverflow question with the correct answer.

As a side node are there any plans to add out of the box support for parallel test runs in Serenity ? 

четверг, 7 сентября 2017 г., 16:02:53 UTC+3 пользователь John Smart написал:

[John Smart]

The problem is, it's not Serenity that doesn't offer parallel support, it's Cucumber. I was thinking of at least writing the approach you describe somewhere, as it seems to be the best at the moment.

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nicolai Baidan]

I could write a small sample project for parallel executions , that could be added to the Serenity repo if it makes sense. 

четверг, 7 сентября 2017 г., 16:29:18 UTC+3 пользователь John Smart написал:

[John Smart]

Certainly :-)

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[SeleniumWhat?]

Hi Nicolai,

 It would be great help if you could spend sometime to create a small demo project describing - in details - how to set up the parallel executions.

 I have thought about it but don't know where to start....:-).

 Please post the link to your demo project so everyone can benefit from it!

 Thanks!

  J.N