CI/CD system with: Selenium, JIRA, Qmetry and Sauce

[Wayne Hazle]

Hello All, I am trying to set up a CI/CD system {and yes I know terminology gets thrown around without people knowing what they are talking about sometines :) } but in short the boss wants daily builds made in DEV and automated scripts run every day with results logged.

For tools we have Selenium & Java, JIRA with plugins for Qmetry and Sauce. The goal is stories written in JIRA associated manual test cases in Qmetry.... Selenium scripts attached in Qmetry (or should that be in Sauce?) . A build is put out a literally a button pressed (or run chosen) and the associated scripts are run, results are logged for the build and read is wonderful esay to read build report.

I am not 100% sure if Sauce needs to be involved. We don't need to run across lots of different platforms. I know they want everything 'in the cloud' so nothing is tied to one specific machine. I would love some thoughts from everyone out there. Are these the right tools? And one of the first steps is should Selenium scripts be stored in Qmetry/JIRA?

I also am not planning to add a framework yet (or should I?). I hate overdoing it with more tools than needed

[Wayne Hazle]

 OK Dev is using Jenkins. Should Selenium scripts live there?

[Krishnan Mahadevan]

Wayne,

 

Your Selenium scripts are supposed to live in a version control system such as Git/SVN/ClearCase etc.,

Since you mention Jira, I am assuming that you anyways would be having an Atlassian license. So maybe you could have your selenium test scripts live in BitBucket.

 

On a side note, where do the dev keep their code? You can have your test code live in the same version control system.

 

In a CI/CD system, Jenkins job would start off by pulling code from a version control system, build the code and run it.

 

Usage of Sauce depends on how your Jenkins environment is setup.  If your Jenkins system is a headless machine (no display configured) and on which there are no browsers installed then you would need a remote execution environment.

 

For e.g., lets say your primary browser to test is Internet Explorer and your Jenkins system runs on LINUX machines, then you would need to have windows boxes available.

You can make it available via either by setting up your own grid or by leveraging a remote execution environment such as SauceLabs.

 

I have only used Jira for defect management and also for user story management. I have not used QMetry so I cannot comment on it.

 

Hope that helps!

 

 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/1c92bbb1-31b9-49de-a715-7fcf918b11c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Oscar Rieken]

thats not really how that all works, your "selenium scripts" are code/tests. so you would have them stored in whatever source code management tool you use.

sauce labs would just be where your tests/code would be executed in a browser/mobile/whatever.

jenkins is a ci tool that you can use to do your builds/run your tests/deploy your code, the only think you would store here would be your build configs (but really you could also save those configs in the same SCM and have it build the jobs dynamically but thats a different conversation)

On Mon, Mar 13, 2017 at 1:41 PM, Wayne Hazle <wayne...@gmail.com> wrote:

 OK Dev is using Jenkins. Should Selenium scripts live there?

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/aa02897a-0258-4d9c-a53e-834978f1c2f0%40googlegroups.com.

[Wayne Hazle]

Thank you both already for your answers.  I am trying too clear through all the fog as I have people evaluating tools and throwing around lots of technologies and titles.

The business need is simple:  Nightly or daily builds and Selenium scripts that run against those build.  Results posted.  Preferably results posted directly to JIRA. 

Dev is storing code in Gitlab.  Jenkins is used to Schedule builds.

I will write scripts in Selenium with JAVA. (have not decided about framework.  for first proof of concept I won't even go down that road)

Selenium scripts storied in Git with code.

Build is made & Selenium scripts should run right after.

Results & reports are generated.

* Reports automatically imported into JIRA (QMetry) where boss can read next morning what has passed and failed.

I think the biggest issue right now is this automatic import.  I think the most important part is getting scripts to run, but everyone is VERY hung up on the import part.  Any thoughts?

On Monday, March 13, 2017 at 10:39:30 AM UTC-7, Wayne Hazle wrote:

Hello All, I am trying to set up a CI/CD system {and yes I know terminology gets thrown around without people knowing what they are talking about sometimes :) } but in short the boss wants daily builds made in DEV and automated scripts run every day with results logged.

[Lodhi]

I'm working on similar requirement, except for it's a little simpler constraints for me: Run Selenium suite nightly after daily build (just a cron job), and report results (by email).

I have recorded test cases through Selenium IDE. To run it from our automated nightly build script, I tried to convert suite (through IDE) to Java/JUnit/Webdriver code but ran into issues like not being able to find an element because a new iFrame was not selected (by IDE) before clicking an element in that new iFrame. So I tried selenese-runner and so far looks good, as it runs the Selenium IDE suite without any changes, through command prompt. Good part about selenese-runner is I get a comprehensive (HTML) report out of the box that I think I should be able to send through email easily.

When you say "import to JIRA", what do u mean exactly? Create JIRA issues based on results? Also, isn't QMetery an automation framework itself, why not use that instead of Selenium or vice versa? 

[Wayne Hazle]

Thank you great answers and questions.  First I understand Qmetry to be framework for Selenium but not an automation tool in and of itself.  So after much back and forth and discussion today if I break down how they envision it:

- We have JIRA and the Qmetry plugin for case management.

- Let us say there is a Test Scenario in Qmetry plug (therefore in JIRA)

         LOGIN-1  Verify that we can log into the system.

- I have a Selenium/Java script that tests this.  I want to be able to attach that script to scenario LOGIN-1 in Qmetry, set up a run "Build 23  3-13-2017",  when the build comes tonight the script runs.  It writes "LOGIN-1 Test Passes" in the run for  "Build 23  3-13-2017" in Qmetry.  When the boss comes in he can see a nice report that shows the tests that ran and the results.

- He prefers it directly in Qmetry rather than me emailing a report.

- I am also debating whether the test should be run from within qmetry or run from Jenkins and just written TO qmetry