Re: [JVM] Cucumber and maven command line

[Martin Busik]

> I have tried all the combinations to run maven with tags. I was just sucessful with tag annonation inside Java class and ant call.
> Tag parameter is not passing by maven. Who was succesful with running specific tag by maven could you please send
> an example of pom and run command please?

Which runner are you using?
We are using the cli runner:

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.2.1</version>
        <executions>
          <execution>
            <phase>integration-test</phase>
            <goals>
              <goal>java</goal>
            </goals>
            <configuration>
              <skip>${skipITs}</skip>
              <classpathScope>test</classpathScope>
              <mainClass>cucumber.cli.Main</mainClass>
              <includePluginDependencies>true</includePluginDependencies>
              <arguments>
                <argument>--tags</argument>
                <argument>~@wip</argument>
                <argument>--tags</argument>
                <argument>@dev</argument>
                <argument>--format</argument>
                <argument>junit:${basedir}/target/output/cucumber-report.xml</argument>
                <argument>--glue</argument>
                <argument>classpath:de.wlps.juli.cucumber</argument>
                <arguments>${basedir}/../spec</arguments>
              </arguments>
            </configuration>
          </execution>
        </executions>
      </plugin>

(which causes the maven proces to terminate, if there were some errors; but this is an another issue, it is good enough to run specific tests without modifying the source code)

[aslak hellesoy]

That's because this setup will run Cucumber in the same JVM as Maven
itself - and Cucumber calls System.exit when it's done.
To avoid that, use exec:exec instead of exec:java:
http://mojo.codehaus.org/exec-maven-plugin/examples/example-exec-for-java-programs.html

Aslak

> --
> -- Rules --
>
> 1) Please prefix the subject with [Ruby], [JVM] or [JS].
> 2) Please use interleaved answers
> http://en.wikipedia.org/wiki/Posting_style#Interleaved_style
> 3) If you have a question, don't reply to an existing message. Start a new
> topic instead.
>
> You received this message because you are subscribed to the Google Groups
> Cukes group. To post to this group, send email to cu...@googlegroups.com. To
> unsubscribe from this group, send email to
> cukes+un...@googlegroups.com. For more options, visit this group at
> https://groups.google.com/d/forum/cukes?hl=en
>
>

[Andrii Dzynia]

Thanks guys, I managed to run it from cli runner. How the maven pom file will be looks like if I would like to use jUnit runner and pass tags from -D parameters?

[Andrii Dzynia]

I got an error when trying to run with different tags

[WARNING] 
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NullPointerException
	at cucumber.formatter.JUnitFormatter.handleHook(JUnitFormatter.java:116)
	at cucumber.formatter.JUnitFormatter.after(JUnitFormatter.java:111)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at cucumber.runtime.Utils$1.call(Utils.java:40)
	at cucumber.runtime.Timeout.timeout(Timeout.java:12)
	at cucumber.runtime.Utils.invoke(Utils.java:36)
	at cucumber.runtime.RuntimeOptions$2.invoke(RuntimeOptions.java:119)
	at $Proxy26.after(Unknown Source)
	at cucumber.runtime.Runtime.runHookIfTagsMatch(Runtime.java:206)
	at cucumber.runtime.Runtime.runHooks(Runtime.java:184)
	at cucumber.runtime.Runtime.runAfterHooks(Runtime.java:179)
	at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:38)
	at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:112)
	at cucumber.runtime.Runtime.run(Runtime.java:105)
	at cucumber.runtime.Runtime.run(Runtime.java:93)
	at cucumber.cli.Main.run(Main.java:20)
	at cucumber.cli.Main.main(Main.java:12)

When I running my tests with jUnit everything looks fine but I cannot pass tag parameter from maven. How I should do this?

Also I was trying to create one more junit runner class RunCuke.java class where differnt tags where specified. But when running "mvn clean test -DRunCuke" tests did not run.

 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[aslak hellesoy]

This looks like a bug. Please file a ticket at github with full
stacktrace and version.

>
> When I running my tests with jUnit everything looks fine but I cannot pass
> tag parameter from maven. How I should do this?
>

When using the JUnit runner, you have to pass the tags with
@Cucumber.Options(tags={"@foo", "@bar"})

This can be overridden by defining the *special* cucumber.options
system property that Cucumber-JVM will parse if defined.
Beware that cucumber.options currently overrides the entire command
line generated from the @Cucumber.Options annotation.

Aslak

[Andrii Dzynia]

Thanks Aslak,

cucumber.options worked fine! Will rise bug today about the previous case

[Sébastien LE CALLONNEC]

Hi Aslak,

On Wed, Sep 5, 2012 at 11h44, aslak hellesoy <aslak.h...@gmail.com> wrote:

>
> When using the JUnit runner, you have to pass the tags with
> @Cucumber.Options(tags={"@foo", "@bar"})
>
> This can be overridden by defining the *special* cucumber.options
> system property that Cucumber-JVM will parse if defined.
> Beware that cucumber.options currently overrides the entire command
> line generated from the @Cucumber.Options annotation.

Interestingly this is something I have recently hit when rewriting my cucumber-jvm example (http://www.weblogism.com/item/334/integration-tests-with-cucumber-jvm-selenium-and-maven) using tags.

Not sure whether this is something you feel strongly about, but I have just submitted a pull request (https://github.com/cucumber/cucumber-jvm/pull/388) to alter that behaviour, as I think it is not correct in the case of junit.  Firstly because the logic for JUnit without system properties is to set default glue and feature paths when none is set -- and setting system properties break that behaviour -- , and secondly (arguably less important, though) because it makes the definition of Maven pom a bit more messy, as you have to specify glue and feature paths which should be "guessed" from the JUnit classes.

Is that behaviour "by design"?

Regards,
Sebastien.

[Andrii Dzynia]

Oh, I back to the post too late. I also tried to fix this issue inside cucumber core. Configure env and just added two lines inside RuntimeOptions. 

When you guys expect to release this fix?

Regards,

Andrii

[aslak hellesoy]

On Fri, Sep 7, 2012 at 4:33 PM, Andrii Dzynia <d3u...@gmail.com> wrote:
> Oh, I back to the post too late. I also tried to fix this issue inside
> cucumber core. Configure env and just added two lines inside RuntimeOptions.
>
> When you guys expect to release this fix?
>

I'd like to get a couple of more fixes in. Next week or two maybe.

[aslak hellesoy]

On Tue, Oct 9, 2012 at 2:46 PM, Vinod Kumar Mohan <mvino...@gmail.com> wrote:
> Hi All,
>
> Is there a fix for running tags in CLI for Cucumber JVM yet.. that can run
> different tags
>

What version?
What options are you using, and in what way doesn't it work for you.

> On Tuesday, September 4, 2012 9:48:36 PM UTC+1, Andrii Dzynia wrote:
>>
>> HI all,

>>
>> I have tried all the combinations to run maven with tags. I was just
>> sucessful with tag annonation inside Java class and ant call. Tag parameter
>> is not passing by maven. Who was succesful with running specific tag by
>> maven could you please send an example of pom and run command please?
>

[Vinod Kumar Mohan]

<dependency>

            <groupId>info.cukes</groupId>

            <artifactId>cucumber-java</artifactId>

            <version>1.0.9</version>

            <scope>test</scope>

        </dependency>

        <dependency>

            <groupId>info.cukes</groupId>

            <artifactId>cucumber-junit</artifactId>

            <version>1.0.9</version>

            <scope>test</scope>

 </dependency>

I am trying to use mvn clean test -Dtags="@Test1"

I want the above maven command to be used for TEst2, Test3 tags at same time in different machines

but unfortunately its picking what i ve given in the Cucumber.Options in RunCukeTest.JAva

Any help on this will be much appreciated

Vinod 

[aslak hellesoy]

On Wed, Oct 24, 2012 at 4:35 PM, Vinod Kumar Mohan <mvino...@gmail.com> wrote:

<dependency>

            <groupId>info.cukes</groupId>

            <artifactId>cucumber-java</artifactId>

            <version>1.0.9</version>

            <scope>test</scope>

        </dependency>

        <dependency>

            <groupId>info.cukes</groupId>

            <artifactId>cucumber-junit</artifactId>

            <version>1.0.9</version>

            <scope>test</scope>

 </dependency>

I am trying to use mvn clean test -Dtags="@Test1"

I want the above maven command to be used for TEst2, Test3 tags at same time in different machines

but unfortunately its picking what i ve given in the Cucumber.Options in RunCukeTest.JAva

Any help on this will be much appreciated

Use -Dcucumber.options

Read History.md for more details. Also search in the issue tracker for cucumber.options.

Aslak

[Vinod Kumar Mohan]

Aslak,

Thanks for the reply,

I am not sure how do i use this DCucumber.Options.

I am using Cucumber JVM with Junit

with out updating the RunCukesTest.Java. Can i execute the  tests from command prompt

like mvn clean test -Dtags= @xxx

i don;t use cucumber.yml. i used Maven 

Cheers

Vinod M

[Joel Byrnes]

I don't think Cucumber-JVM supports cucumber.yml, that's a ruby thing. 

I think I have a better solution for you than running from command line and specifying tags. Use tags specified in the junit test annotation. 

SpecialTest.java:

@RunWith(Cucumber.class)

@Cucumber.Options(tags = "@special")

public class SpecialTest { }

That will run all your scenarios that have the @special annotation. You haven't provided any source, so you need to put in the correct features and other Options in. Then you run just that one test with

mvn test -Dtest=SpecialTest

If this test should not be run with other tests, say for example your other tests specifically exclude @special tests (with tags = "~@special") then you can exclude this test from running by default. In your maven surefire config in build:

            <plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-surefire-plugin</artifactId>

                <version>${maven-surefire-plugin.version}</version>

                <configuration>

                    <excludes><exclude>**/SpecialTest.java</exclude></excludes>

                </configuration>

            </plugin>

FYI you can also "include" specific tests; you could make it by default inclusive, or exclusive. 

I haven't actually tested the above build config; you may have to do something with profiles to be able to run different test classes. In fact that might be better: a maven profile that just runs that one test class, overriding the default behaviour. 

Hope this helps. 

[Vinod Kumar Mohan]

Hi Joel,

Thanks for the response. Apologies for not explaining my situation well enough. 

I have a framework which has basically three types of Test Scripts - API Tests, Web Tests and a Salesforce AppTests.. Each has their own  annotations ( for eg; API - @API, WEB - @WEB, Salesforce - @SF) 

We were running these tests indepedently against different builds in Jenkins CI,

Previously we were using Cuke4Duke , which allows us to run different tags using Junit Runner. (mvn clean test -Dtags = "@WEB" or "@API" or "@SF"

Since when we migrated to Cucumber JVM we are not able to do so. how ever i can use ur solution given below to avoid WIP tests. 

Any help on this would be very much appreciated

Thanks 

V

[Roberto Lo Giacco]

Il giorno giovedì 25 ottobre 2012 08:26:18 UTC+1, Joel Byrnes ha scritto:

I don't think Cucumber-JVM supports cucumber.yml, that's a ruby thing. 

correct

 

I think I have a better solution for you than running from command line and specifying tags. Use tags specified in the junit test annotation. 

SpecialTest.java:

@RunWith(Cucumber.class)

@Cucumber.Options(tags = "@special")

public class SpecialTest { }

That will run all your scenarios that have the @special annotation. You haven't provided any source, so you need to put in the correct features and other Options in. Then you run just that one test with

mvn test -Dtest=SpecialTest

If this test should not be run with other tests, say for example your other tests specifically exclude @special tests (with tags = "~@special") then you can exclude this test from running by default. In your maven surefire config in build:

 

            <plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-surefire-plugin</artifactId>

                <version>${maven-surefire-plugin.version}</version>

                <configuration>

                    <excludes><exclude>**/SpecialTest.java</exclude></excludes>

                </configuration>

            </plugin>

FYI you can also "include" specific tests; you could make it by default inclusive, or exclusive. 

I haven't actually tested the above build config; you may have to do something with profiles to be able to run different test classes. In fact that might be better: a maven profile that just runs that one test class, overriding the default behaviour. 

 

This is a bad idea, mainly because you will either need to have one specific test class and maven profile for each tag combination you want to support OR have to change the source code and recompile every time you want to change the configuration.

The best solution is the one suggested by Aslak which is to provide a sensible set of default values and then override them on a case by case basis providing command line options.

have a look at https://groups.google.com/d/msg/cukes/iQZJC5PyUhA/t8Qa0HLhoggJ

With that solution you can decide at runtime if you want to run only features/scenarios tagged as `special` just executing

mvn verify -Dtags=@special

And if you want to run `special` and `sprint-1` you can issue

mvn verify -Dtags=@special,@sprint-1

I'm trying to set up a complete example project on BitBucket, I'll post here when it's ready. 

[Roberto Lo Giacco]

I think you want to have a look at the sample Maven project I pushed on BitBucket https://bitbucket.org/rlogiacco/atdd

It provides support for:

• running Cucumber tests within Eclipse (JUnitWrapper)
• running Cucumber tests in a completely automated fashion (starts tomcat, runs the tests, stops tomcat)
• generate unit and automated (Cucumber) test coverage reports (using Jacoco)
• supports Gherkin tags with the -Dtags= parameter
• supports single feature files execution (to help developers) with the -Dfeature= parameter

Please let me know if you find it useful

[Joel Byrnes]

I don't think it's a bad idea, merely one way of choosing what to run, considering the command line options through maven are currently somewhat limited. You can select scenarios to run in a number of ways:

* tags - include and exclude

* feature file

* dir of feature files

* Junit files specifying any of the above

* (and now I know about) cucumber.options overriding junit annotations

So the method(s) you choose will depend on build environment, needs and habits. If you use tags, you might end up with an overwhelming number of tags on each scenario, which would become hard to maintain - personally I use tags as sparingly as possible. If you break up features into dirs, you need to specify the dirs in some way, eg junit or command line. If you do everything through the command line, this might make maintaining multiple build profiles hard in your CI configuration. If you do it through Junit config, then as you say you have to recompile them (slow with maven), but it's easy to configure a CI build. 

I don't find it slow to configure things in Junit tests, because I avoid using maven command-line and run them in the IDE the majority of the time, so there is very little "compile time". However, it would be even better if there was better IDE support for features, so you could directly run a feature file, and individual scenarios from the result. I believe JetBrains/IntelliJ is working on that now. 

So when you say to change command line options on a case-by-case basis it depends how many different cases you have and how often you change them. 

That said, it would be nice if all my cucumber results could be output all in the same html/junit/json file, instead of the individual junit tests I have them in now, which select directories of features to run (of related functionality). I will look at ways to make use of this command-line option. 

[Joel Byrnes]

Thanks, I'll have a look. 

[Joel Byrnes]

Oh, hey look at this from the 1.1 announcement: 

[Core] The cucumber.options System property will no longer completely override all arguments set in @Cucumber.Options or on the command line. Instead, it will keep those and only override those that are specified in cucumber.options. Special cases are --tags, --name and path:line, which will override previous tags/names/lines. To override a boolean option (options that don't take arguments like --monochrome), use the --no- counterpart (--no-monochrome). (#388 Sébastien Le Callonnec, Aslak Hellesøy)

So that solves some of my issues with the command line option :)

[aslak hellesoy]

On Wed, Oct 24, 2012 at 4:59 PM, Vinod Kumar Mohan <mvino...@gmail.com> wrote:

Aslak,

Thanks for the reply,

I am not sure how do i use this DCucumber.Options.

Do -Dcucumber.options="--tags @xxx" instead of -Dtags=@xxx

Make sure you're using version 1.1.1 - this is a new feature.

Aslak

I am using Cucumber JVM with Junit

with out updating the RunCukesTest.Java. Can i execute the  tests from command prompt

like mvn clean test -Dtags= @xxx

i don;t use cucumber.yml. i used Maven 

Cheers

Vinod M

On Tuesday, September 4, 2012 9:48:36 PM UTC+1, Andrii Dzynia wrote:

HI all,

I have tried all the combinations to run maven with tags. I was just sucessful with tag annonation inside Java class and ant call. Tag parameter is not passing by maven. Who was succesful with running specific tag by maven could you please send an example of pom and run command please?

--

[Vinod Kumar Mohan]

Hi Aslak,

I tried DCucumber.Options and specified tags, This option is not over riding the tag that i have given in the RunCukesTest.Java files.

The scenario is

i am using the line mvn clean test -DCucumber.options"--tags=@WEB11"

but in the RunCukesTest.JAVA, 

import cucumber.api.junit.Cucumber;

import org.junit.runner.RunWith;

@RunWith(Cucumber.class)

@Cucumber.Options(format = {"pretty", "html:target/cucumber", "json:target/cucumber.json"},tags={"@WEB"})

public class RunCukesTest {

}

the Tests that are running are with the tag "WEB" not "WEB11". 

I am not sure what else i need to do with this. 

P.S I am usning Cucumber JVM 1.1.1

Any help on this will be much appreciated

Thanks for your continuous assistance

Regards

Vinod M 

[aslak hellesoy]

On Thu, Nov 8, 2012 at 3:02 PM, Vinod Kumar Mohan <mvino...@gmail.com> wrote:

Hi Aslak,

I tried DCucumber.Options

Java system properties are case sensitive. It's -Dcucumber.options

[Vinod Kumar Mohan]

Hi Aslak,

Thanks for that, but this time i am getting different error looks like mvn is not taking that tag -Dcucumber.options .the stack trace is as below;

C:\_Automation\Driving\Hotrod-QA-Automation>mvn clean test Dcucumber.options"--t

ags=@WEB11"

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building Cucumber-JVM 1.0.0

[INFO] ------------------------------------------------------------------------

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 0.383s

[INFO] Finished at: Thu Nov 08 15:56:18 GMT 2012

[INFO] Final Memory: 4M/91M

[INFO] ------------------------------------------------------------------------

[ERROR] Unknown lifecycle phase "Dcucumber.options--tags=@WEB11". You must speci

fy a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or <pl

ugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecyc

le phases are: validate, initialize, generate-sources, process-sources, generate

-resources, process-resources, compile, process-classes, generate-test-sources,

process-test-sources, generate-test-resources, process-test-resources, test-comp

ile, process-test-classes, test, prepare-package, package, pre-integration-test,

 integration-test, post-integration-test, verify, install, deploy, pre-site, sit

e, post-site, site-deploy, pre-clean, clean, post-clean. -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit

ch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please rea

d the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/LifecyclePhase

NotFoundException

Any thoughts abt this !!

Thanks 

Vinod 

[Vinod Kumar Mohan]

Apologies.... i missed out the "-" in it

[Vinod Kumar Mohan]

Thanks a lot ASLAK....