How to test an individual capability or an individual feature from the command line?

[Kevin Pauli]

I have a class that extends ThucydidesJUnitStories, which by default matches all my JBehave stories under src/test/resources/stories.  Under "stories" I have a folder for each capability, and then under there I have folders for each feature.

I would like to set some property on the command line (metafilter perhaps?) that, when I run the class in my IDE, will run only stories in one particular capability folder, or one particular feature folder.

I realize that I could write whole new java class and override a method to make it only find certain stories, but I do NOT want to do that.  These are adhoc runs via the IDE during development, and it is burdensome to be creating such classes all the time.

I'm thinking there's got to be a way to do it from the command line.  After extensive googling I just can't seem to find the right property name or metafilter expression.

[John Smart]

From the command line, you could use JBehave tags, or write a class for each JBehave story (extending ThucydidesJUnitStory) so that you can run those arbitrarily. The second approach also makes it easier to run stories in parallel (which JBehave natively does very sloppily).

--
You received this message because you are subscribed to the Google Groups "Thucydides Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send an email to thucydid...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

___________________________________________________
John Smart | Wakaleo Consulting  |  +61 407 247 642
Optimizing your software development process
http://www.wakaleo.com  |  john....@wakaleo.com
___________________________________________________

The dates for the 2014 BDD workshops have been scheduled! Check out our upcoming BDD/TDD Master classes and our Advanced BDD Requirements Workshops, coming soon to Sydney and Melbourne!

___________________________________________________

[Kevin Pauli]

I appreciate the quick response.  However I do not want to write classes for each story.  

The main reason is that the .story files are written by BAs (Business Analysts).  The BAs need to be able to move stories around and rename the folders, as they refine the concepts of the Capabilities and Features.  If the paths to the folders are hardcoded in a Java class file, the tests won't be found and they will break when the CI server runs the build after the BA commits the changes.  That's why I have the single, top-level ThucydidesJunit test class; when the CI build happens, it finds and runs everything regardless of the capability or feature folder.  The BAs can run wild.

Furthermore, such one-off classes end up being just boilerplate that ultimately just points at a folder.  The meat is in the .story files.  There is nothing at all interesting in them other than a path to a folder; this is begging to be a system property of some kind.

Can you tell me more about the JBehave tag approach you mentioned?  I did consider putting tags into each story identifying the feature and capability, but then didn't because that seems insane.  Each story is already in a folder that defines the feature and capability.  I shouldn't have to tell it again.  It violates DRY principle and places an additional burden on the BAs who might want to refactor the Capabilities and Features concepts of our product, to keep all the tags and the folder names in sync!

[Shawn Boyle]

There seems to be a lot of discussion in this forum recently regarding using tags within tests and, quite frankly, I still get the sense that the Serenity user community still does not fully grasp using tags (including myself). I think adding some documentation that describes how to use the feature will go a long way (I'd submit a pull request myself if I can figure out how to get tags/filtering working):
http://thucydides.info/docs/serenity-staging/#_running_scenarios_by_tags

A motivating feature to use Serenity is its flexibility to support multiple BDD tools (Cucumber, JBehave, etc) and build tools (maven, gradle, etc) but this flexibility has a consequence of increased complexity because the documentation needs to support the various permutations (JUnit, Cucumber, JBehave and maven, gradle, etc):

• The @tag (or @tags) seems to be a Serenity-specific tag so it's unclear if it's possible to run Serenity to filter to only run tests that have a specified tag value?
  
• I agree with Kevin's thoughts in that it seems logical that a user should be able to run tests filtered by a Capability, Feature, tag, etc. That being said, from the BA perspective, the BA only cares that all of the features are there; he/she will not likely want to run a subset of the capabilities/features (this seems more of a developer need).
  
• Since Serenity wraps Selenium some documentation (and perhaps it's already there) should include which features/options can be used and how do I configure the Selenium options via Serenity? 
• In trying to setup Serenity to run as part of an automated acceptance test stage within a Continuous Delivery workflow (using Bamboo and Gradle), I've run into the issue of trying to run Serenity on a "headless workstation". I've had a bit of success using Xvfb (on Linux) but I have not yet been able to run it from a Bamboo instance (the gradle check aggregate command seems to hang the bamboo agent).
  

Really, I'm trying to understand the preferred way to use tags using JBehave and gradle. I have not had any luck getting it to work (I want to be able to only run tests/stories/scenarios that have a specific tag assigned to them); so my .story has something like:

Meta:
@tag Release: 1.0
@testtype integration

And then I've tried running the following gradle command:

gradle clean check aggregate -Dmetafilter="+testtype integration"

Again, I think it's probably just a matter of getting the right amount of documentation and it will go along way. I'm trying to be an advocate for using Serenity within my company but these the tag/filter/running headless mode stumbling blocks makes it hard to continue to recommend using Serenity on project across my company.

Any feedback suggestions appreciated!

Shawn

[Theo]

Please check Chapter 13 in Serenity Documentation: https://github.com/serenity-bdd/serenity-documentation/blob/master/src/asciidoc/tags.adoc

(you might want to fetch the project and build it locally to get it nicely as a html file)

I will have a look if there are areas not covered by documentation and try to improve it, wherever possible

Cheers

Theo

[Shawn Boyle]

Thanks for the response, Theo.

I have seen the documentation that you referenced. I did a bit more investigation and I believe there is an issue in the gradle implementation; the -Dmetafilter option doesn't get passed to Serenity because Gradle forks a new JVM and doesn't seem to pass the initial JVM args to the forked JVM. I have a couple of workarounds:

• I can put a the metafilter property in the serenity.properties 
• I can set a (Linux) system environment variable that I can access within the my Serenity bootstrap class that extends SerenityStories
  

What I ended up doing (for now) is updating the build.gradle that runs Serenity to use gradle project properties (which are passed in via the -P option). We already had a task that copies an environment-specific serenity.properties file to the gradle projectDir and then I just append a property to the copied properties at runtime:

task copyPropsFile << {
    if(!project.hasProperty('environment')){
        ext.environment = 'dev'
    }
           
    copy{
      from '../conf/' + environment + '/properties/serenity.properties'
      into projectDir
    }
   
    // Appends the JBehave metafilter property to the property file
    // See JBehave metafilter documentation: http://jbehave.org/reference/stable/meta-filtering.html for metafilter options    
    if (project.hasProperty('metafilter')) {
      println "JBehave metafilter set to: $metafilter"
     
       ant.propertyfile(file: "$projectDir/serenity.properties") {
         entry( key: "metafilter", value: "$metafilter")
      }        
    }    
}

// Hook into the gradle processTestResources task to execute the copyPropsFile custom task
processTestResources{
    doFirst{
        copyPropsFile.execute()
      }
}

Ideally, it would be nice to use the -Dmetafilter option but for now the gradle -Pmetafilter="+SomeMetaTag SomeMetaTagValue" does the trick for us.

[Michael D D'Amato]

Very helpful! Thanks for this post!

[Dmitriy Grablyov]

Hi, Kevin.

You wrote that your acceptance test class extends ThucydidesJUnitStories and runs all the stories from the folder by default.

You can modify it a little bit and pass the path to the stories as a property.

So if you do not specify any property - let Thucydides run all the stories by default. And if you specify that property - override that behavior.

In the class initialization block call findStoriesCalled(path_to_the_stories);

In this case note, that Thucydides understands wildcard paths. If you need then to run specific capability, just pass src/test/resources/stories/capability_folder_name/**/*.story

for a specific feature src/test/resources/stories/capability_folder_name/feature_folder_name/**/*.story or even easier **/feature_folder_name/**/*.story

Regards,

Dmytro