Pending/Skipping Scenarios in Serenity Cucumber

[Santosh Pillai]

Hi,

I'm using Serenity with Cucumber. I've a feature file with some scenarios. I want to able to skip some of these scenarios, but its not working for me. 

I tried doing, 

@Pending

Scenario

Step1

Step2...

@Skip

Scenario

Step1

Step2..

@Manual

Scenario

Step1

Step2..

Can anyone please help me out if I'm doing it wrong and what's the right way to implement it. Thanks

[Tony Tubbs]

I am seeing the same issue.  Following is my feature file, simple 1 + 1 = 4 failing test to verify tags are working properly with cucumber.  Attached screen shots show results.  The failed assertion shows in all cases, indicating the tags are not being processed as expected.  Please advise.

@example @verify_tags

Feature: Verify serenity tag usage

Verify serenity tags work with cucumber

  @ignore

  Scenario: Use of @ignore tag

    Given a number '1'

    And a number '1'

    When added together

    Then the result is '4'

  @ignored

  Scenario: Use of @ignored tag

    Given a number '1'

    And a number '1'

    When added together

    Then the result is '4'

  @pending

  Scenario: Use of @pending tag

    Given a number '1'

    And a number '1'

    When added together

    Then the result is '4'

  @skip

  Scenario: Use of @skip tag

    Given a number '1'

    And a number '1'

    When added together

    Then the result is '4'

  @wip

  Scenario: Use of @wip tag

    Given a number '1'

    And a number '1'

    When added together

    Then the result is '4'

  @manual

  Scenario: Use of @manual tag

    Given a number '1'

    And a number '1'

    When added together

    Then the result is '4'

[John Smart]

What version of Serenity are you using?

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

___________________________________________________
John Smart | Wakaleo Consulting  |  +44 7398 832273
Making smart teams collaborate better
http://johnfergusonsmart.com  |  john....@wakaleo.com
___________________________________________________

We love breaking down silos and helping smart teams collaborate better! Ask about our tailored on-site workshops in Agile Product Planning, BDD Requirements Discovery,  BDD, TDD and Clean Coding, and Advanced BDD Test Automation.

___________________________________________________

[Tony Tubbs]

    <properties>

        <jdk.version>1.7</jdk.version>

        <slf4j.version>1.7.7</slf4j.version>

        <serenity.version>1.1.37-rc.7</serenity.version>

        <serenity.maven.version>1.1.37-rc.7</serenity.maven.version>

        <serenity.cucumber.version>1.1.10</serenity.cucumber.version>

        <parallel.tests>4</parallel.tests>

    </properties>

[John Smart]

Could you try with a more recent version (e.g. 1.1.40)?

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tony Tubbs]

Updated to use following versions.  Made no difference in output.

<serenity.maven.version>1.1.40</serenity.maven.version>

<serenity.cucumber.version>1.1.10</serenity.cucumber.version>

[John Smart]

Could you produce a sample produce to reproduce the issue? I tried with your feature file but without getting the outcomes you described.

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-users+unsubscribe@googlegroups.com.
To post to this group, send email to thucydides-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tony Tubbs]

Find sample project here:

https://github.com/tbbstny/SerenityCucumberStudy

[Tony Tubbs]

Updated the sample app so tests can be ran during the build (in order to eliminate the scanning of test-jar as the problem)

mvn clean install -PSerenity

Sample app here: https://github.com/tbbstny/SerenityCucumberStudy

[Tony Tubbs]

Did the sample app help?  Can you confirm if this is an issue or not?

[John Smart]

Thanks Tony,

We were able to reproduce the issue. It will be fixed in the next release. The problem was caused by the assert statements in the step definitions, not inside nested steps or Screenplay consequences. This means that Serenity has no way of not executing the code even if you use a @pending or @Ignore annotation - instead it will refrain from executing any steps or Screenplay tasks, and mark the result as ignored/skipped etc. But if you run the test in an IDE, you will still see the exception.

-- 
___________________________________________________
John Smart | Wakaleo Consulting  |  +44 7398 832273
Making smart teams collaborate better
http://johnfergusonsmart.com  |  john....@wakaleo.com
___________________________________________________

We love breaking down silos and helping smart teams collaborate better! Ask about our tailored on-site workshops in Agile Product Planning, BDD Requirements Discovery,  BDD, TDD and Clean Coding, and Advanced BDD Test Automation.
___________________________________________________

On 28 Sep. 2016, at 4:05 pm, Tony Tubbs <ttu...@gmail.com> wrote:

Did the sample app help?  Can you confirm if this is an issue or not?

--
You received this message because you are subscribed to the Google Groups "Serenity BDD Users Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send email to thucydid...@googlegroups.com.

[Tony Tubbs]

Thanks for the update!

Just to make sure I understand correctly, are saying you will be providing a fix that will in fact NOT run the cucumber step definitions? Once that fix is in place, my assertions in the cucumber step definitions will no longer be executed? 

Or, is there cleanup needed on my side (also)?  Like moving the assertions down into a Serenity derived type?

[Tony Tubbs]

I moved my cucumber step definition code into a Step Library, making all my Step Definitions a simple pass through to the Step Library.  

@wip, @skip, @pending, @manual all working now

@ignore, @ignored both execute the Step Definition and Step Library code (i.e. neither is getting ignored)

[Tony Tubbs]

Also, because the @ignore was still executed, my report says 0 ignored, but also showing 1 pending.  That means @wip and @skip are not counted on the report result summary.  Should they be?