cucumber plugin request

[Alexey Melezhik]

Hi! I want to add this to jenkins plugins list. Cucumber jenkins plugin to run cucumber features through jenkins.

https://github.com/melezhik/cucumber-plugin

Regards.

Alexey

[Kanstantsin Shautsou]

What is the main goal of this plugin?

1) Usually everybody run cucumber from native packagers/tools and then just grab test results in i.e. https://wiki.jenkins-ci.org/display/JENKINS/Cucumber+Test+Result+Plugin

2) part of commit messages are in russian, not really convenient for contributing by other people

3) https://github.com/melezhik/cucumber-plugin/blob/master/models/cucumber-builder.rb#L65-L67

If you want wrote your commands in ruby, then you can just use shell execution step with #!/bin/env ruby and your script or install some plugin that can highlight ruby syntax. All input variable can be define as input strings or just in this shell script...

Imho this plugin looks absolutely useless for hosting at infra, few shell commands implemented like a plugin will be unreally difficult to support. (Note, it just my opinion.)

[melezhik]

Kanstantin, thank you for your opinion. My replies your comments.

| 1) Usually everybody run cucumber from native ....

you do not need to grab test results if you use this plugin, you just run it as is, and have builder step failed or pass whether your cucumber tests are ok or not. 

| 2) part of commit messages are in russian, not really convenient for contributing by other people

it's not a problem at all, if someone clear some details one may as in English, other comments of course may be written in English in case of any collaboration, right now I just cannot see any ...)))   

| 3) If you want wrote your commands in ruby, then you can just use shell execution step with #!/bin/env ruby 

this is about implementation, plugin users should not care about this, but if to go into technical dispute  I cannot see strong difference between your approach ( use shell scripts , or other "shell" plugins ) and mine ( using system() calls inside ruby ). 

Well, one day I even may choose better tools to handle IPC stuff, like ruby open3 module, or something like that; right now  system() way  is straightforward approach, but that is seemed enough.

| Imho this plugin looks absolutely useless for hosting at infra, few shell commands implemented like a plugin will be unreally difficult to support.

so many thing can be expressed in few lines of bash code, is not it?  ))) , but this  definitely breaks encapsulation principal , while plugins give us freedom to change implementation while stick to public interface  ...

 

вторник, 16 декабря 2014 г., 17:00:56 UTC+3 пользователь Kanstantsin Shautsou написал:

[James Nord]

Hi all,

The plugin does not run cucumber but a specific (ruby) implementation. 

Therefore if this plugin was to be forked I would prefer that either

1) it would in future accept pull request to run other cucumber implementations
or
2) it would be remanbed to cucumber-ruby-builder-plugin (or something more specific).

Also I believe the name cucumber-plugin is too generic - there are already other cucumber plugins out there - this is specifically a cucumber  builder.

I also think it seems to be specific to the OPs need - in that it assumes the use of a browser (chrome/firefox) and a display - neither of which are required for running cucumber (jvm at least) and assumes running on Linux, requires the use of profiles (I believe if you don;t have a cucumber profile file then cucumber will barf when trying to load it) etc.

| 1) Usually everybody run cucumber from native ....

you do not need to grab test results if you use this plugin, you just run it as is, and have builder step failed or pass whether your cucumber tests are ok or not.

Cucumber will (should?!?) return an exit code to say if it passed or failed - so you don't have to parse the json output etc to know if some tests failed.
However what is the point in knowing some tests failed if you don't know what failed?

If you just want to ignore test failures then on linux/unix you can do this from native just run "set +e" if your bash script before invoking the native tool.  (or use 'command || echo "test failed" ' syntax).

/James

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/a0b01d0f-e91b-4516-abfb-c32a6cfec4ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[melezhik]

Hi  James! and thanks for feedback ...

Hi all,

The plugin does not run cucumber but a specific (ruby) implementation. 

Therefore if this plugin was to be forked I would prefer that either

1) it would in future accept pull request to run other cucumber implementations
or
2) it would be remanbed to cucumber-ruby-builder-plugin (or something more specific).

that's fine, no problem,  indeed the plugin implies it runs cucumber under bundle exec , I do not mind to rename it to something proper ....

 

Also I believe the name cucumber-plugin is too generic - there are already other cucumber plugins out there - this is specifically a cucumber  builder.

see my previous comment, cucumber-builder - that's fine 

 

I also think it seems to be specific to the OPs need - in that it assumes the use of a browser (chrome/firefox) and a display - neither of which are required for running cucumber (jvm at least) and assumes running on Linux, requires the use of profiles (I believe if you don;t have a cucumber profile file then cucumber will barf when trying to load it) etc.

- display_ID is optional, you do not have to set it and may live it blank, sorry the screenshot on doc page is outdated, display_ID in last version is in advanced settings and is not shown by default ... 

- it does not assume any type of tests but cucumber, selenium . chrome, firiefox, whatever - the plugin agnostic about this ...

| requires the use of profiles 

not a big deal , I think in _most_ cases people use at least _default_ profile, is not it? anyway, it's not that hard to make profile parameter optional ... ( next versions )

 

 

| 1) Usually everybody run cucumber from native ....

you do not need to grab test results if you use this plugin, you just run it as is, and have builder step failed or pass whether your cucumber tests are ok or not.

Cucumber will (should?!?) return an exit code to say if it passed or failed - so you don't have to parse the json output etc to know if some tests failed.
However what is the point in knowing some tests failed if you don't know what failed?

 

indeed you know! the plugin shows report status in default cucumber output format ( shows  which features fails ) , and it's even easier to make report output format  configurable ( default, progressive , ... ) in next plugin versions ....

 

 

If you just want to ignore test failures then on linux/unix you can do this from native just run "set +e" if your bash script before invoking the native tool.  (or use 'command || echo "test failed" ' syntax).

not show in plugin documentation but one may check  "ignore failures" option to let jenkins go to next bulder step even though cucumber tests run in plugins fail ... plugin just would write test.fail in workspace   directory - to be accessible in next builders .  

[melezhik]

it would in future accept pull request to run other cucumber implementations

you are welcome! or course ...  

[James Nord]

On 19/12/2014 11:52, melezhik wrote:

it would in future accept pull request to run other cucumber implementations

you are welcome! or course ...  

I fall into the same crowd as Kanstantin - in that I use the native launcher. 

Not quite true - I bundle the native launcher (cucumber-jvm) and all my code into a single executable jar - then use "java -jar myTests.jar <insert cucumber args here>").

/James

[melezhik]

пятница, 19 декабря 2014 г., 15:14:49 UTC+3 пользователь James Nord написал:

On 19/12/2014 11:52, melezhik wrote:

it would in future accept pull request to run other cucumber implementations

you are welcome! or course ...  

I fall into the same crowd as Kanstantin - in that I use the native launcher. 

what do you mean? what crowd?

 

Not quite true - I bundle the native launcher (cucumber-jvm) and all my code into a single executable jar - then use "java -jar myTests.jar <insert cucumber args here>").

So , you use cucumber-jvm  and run  "java -jar myTests.jar" through jenkins? Did I get you right? 

 

/James

[James Nord]

On 19/12/2014 12:21, melezhik wrote:

пятница, 19 декабря 2014 г., 15:14:49 UTC+3 пользователь James Nord написал:

On 19/12/2014 11:52, melezhik wrote:

it would in future accept pull request to run other cucumber implementations

you are welcome! or course ...  

I fall into the same crowd as Kanstantin - in that I use the native launcher. 

what do you mean? what crowd?

Sorry - it's an English colloquialism.  I'm in the group of people that just use the native launcher and grab the results - see original comment below.

"Usually everybody run cucumber from native packagers/tools and then just grab test results in i.e. https://wiki.jenkins-ci.org/display/JENKINS/Cucumber+Test+Result+Plugin"

 

Not quite true - I bundle the native launcher (cucumber-jvm) and all my code into a single executable jar - then use "java -jar myTests.jar <insert cucumber args here>").

So , you use cucumber-jvm  and run  "java -jar myTests.jar" through jenkins? Did I get you right? 

Yes.
We have a job that builds and packages the features and glue code (and runs unit tests against the glue code) - then when we want to run the tests against a system we just grab the specific binary we want from Jenkins and then execute it with the normal cucumber args.
(we can also use the same single jar from outside Jenkins to test any arbitrary installation)

[melezhik]

пятница, 19 декабря 2014 г., 15:43:25 UTC+3 пользователь James Nord написал:

that's fine James, that make a sense ...  But what about those who want just to run features placed somewhere in a directory? this plugin may be interesting ...  no packaging, no glue code, no converting into jar, no intermediate operations,  just ... checkout from SCM into workspace using GIT/SVN plugin and run features as is ... , but I assume that there are many ways to do this , and java/cucumber-jvm approach match better for java oriented applications  ...