HP Quality Center

[George_C]

Any one testing with Cucumber but using HP Quality Center to organize
the requirements and tests? If so, how?

[QA Manager]

On Wednesday, December 21, 2011 9:50:04 AM UTC-5, George_C wrote:

Any one testing with Cucumber but using HP Quality Center to organize
the requirements and tests? If so, how?

It appears that no one was driving Cucumber from HP Quality Center (HPQC, now called ALM) when George posted this question, but I want to make it happen now. Ideally, I think the data flow looks like this:

   1) Business Analyst puts requirements into the requirements tool (ideally including some requirements as BDD test scenarios).
   2) Requirements get synchronized into HPQC/ALM.
   3) QAs write more BDD test scenarios (as HPQC test cases) and link them to requirements.
   4) All BDD test scenarios are synchronized into Cucumber. Synchronization includes the HPQC test case ID (unique number).
   5) Someone runs test scenarios, either from HPQC or from Hudson.
      a. In HPQC we could collect a number of test scenarios into a single group and then run the entire group as a batch. In the general case this would require an immediate synchronization, because there could be brand new scenarios (not previously synced to Cucumber) or the grouping could be brand new.
      b. From Hudson we would run a pre-defined batch; these might not change often.
   6) All results get stored in HPQC under the correct test case (based on the test case ID).

HPQC has an API (called OTA) that can handle this; the key is attaching the test case ID to the test case, and extending Cucumber to call that API to record the Pass/Fail result for each test case ID.

I have written a lot of code, but am a newbie with Cucumber and Ruby (we will use Java). So, questions for the community:
   1) What is the best way to attach the test case ID to the test case?
       - We could add a new 'TestCaseID' keyword to Gherkin, and that wouldn't be specific to HPQC; it would support any test management framework.
       - What other choices are there? Is one of them clearly better?
   2) How do we extend Cucumber to call the HPQC API and record the test results?

I want to create something that will be useful to the broad community rather than something that meets my needs and only my needs. Who can point me in the right direction?
 

[Tim Walker]

[Tim] Below...

On Mon, Jan 13, 2014 at 10:00 AM, QA Manager <no_ca...@yahoo.com> wrote:

On Wednesday, December 21, 2011 9:50:04 AM UTC-5, George_C wrote:

Any one testing with Cucumber but using HP Quality Center to organize
the requirements and tests? If so, how?

It appears that no one was driving Cucumber from HP Quality Center (HPQC, now called ALM) when George posted this question, but I want to make it happen now. Ideally, I think the data flow looks like this:

   1) Business Analyst puts requirements into the requirements tool (ideally including some requirements as BDD test scenarios).
   2) Requirements get synchronized into HPQC/ALM.
   3) QAs write more BDD test scenarios (as HPQC test cases) and link them to requirements.

[Tim] Would urge you to do this collaboratively with PM/DEV.  

   4) All BDD test scenarios are synchronized into Cucumber. Synchronization includes the HPQC test case ID (unique number).
   5) Someone runs test scenarios, either from HPQC or from Hudson.
      a. In HPQC we could collect a number of test scenarios into a single group and then run the entire group as a batch. In the general case this would require an immediate synchronization, because there could be brand new scenarios (not previously synced to Cucumber) or the grouping could be brand new.
      b. From Hudson we would run a pre-defined batch; these might not change often.
   6) All results get stored in HPQC under the correct test case (based on the test case ID).

HPQC has an API (called OTA) that can handle this; the key is attaching the test case ID to the test case, and extending Cucumber to call that API to record the Pass/Fail result for each test case ID.

I have written a lot of code, but am a newbie with Cucumber and Ruby (we will use Java). So, questions for the community:
   1) What is the best way to attach the test case ID to the test case?
       - We could add a new 'TestCaseID' keyword to Gherkin, and that wouldn't be specific to HPQC; it would support any test management framework.
       - What other choices are there? Is one of them clearly better?
   2) How do we extend Cucumber to call the HPQC API and record the test results?

I want to create something that will be useful to the broad community rather than something that meets my needs and only my needs. Who can point me in the right direction?
 

--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Leslie Brooks]

Opinions on the best way to implement this? Tags? Hooks? Tags and Hooks?

[Vincent Daburon]

Hi,

The best way for java application to communicate with HP QC is now the

“ HP ALM Synchronizer Adapter Software Development Kit “

https://hpln.hp.com/page/hp-alm-synchronizer-adapter-software-development-kit-0

not the "old" OTA library.

Regards.
Vincent D.

[Vincent Daburon]

Hi,

I think you could use specific tags like @TestCaseID_12345 to add information or relation between the Test Case and the gherkin scenario or the gherkin feature.
@TestCaseID_xxxx

and add the Test Case Name likes
@TestCaseName_E01010 @TestCaseName_<Name of the Test Case>

Regards
Vincent D.

[Matt Wynne]

Hi Leslie,

On 13 Jan 2014, at 17:00, QA Manager <no_ca...@yahoo.com> wrote:

On Wednesday, December 21, 2011 9:50:04 AM UTC-5, George_C wrote:

Any one testing with Cucumber but using HP Quality Center to organize
the requirements and tests? If so, how?

It appears that no one was driving Cucumber from HP Quality Center (HPQC, now called ALM) when George posted this question, but I want to make it happen now. Ideally, I think the data flow looks like this:

   1) Business Analyst puts requirements into the requirements tool (ideally including some requirements as BDD test scenarios).
   2) Requirements get synchronized into HPQC/ALM.
   3) QAs write more BDD test scenarios (as HPQC test cases) and link them to requirements.
   4) All BDD test scenarios are synchronized into Cucumber. Synchronization includes the HPQC test case ID (unique number).
   5) Someone runs test scenarios, either from HPQC or from Hudson.
      a. In HPQC we could collect a number of test scenarios into a single group and then run the entire group as a batch. In the general case this would require an immediate synchronization, because there could be brand new scenarios (not previously synced to Cucumber) or the grouping could be brand new.
      b. From Hudson we would run a pre-defined batch; these might not change often.
   6) All results get stored in HPQC under the correct test case (based on the test case ID).

HPQC has an API (called OTA) that can handle this; the key is attaching the test case ID to the test case, and extending Cucumber to call that API to record the Pass/Fail result for each test case ID.

I have written a lot of code, but am a newbie with Cucumber and Ruby (we will use Java). So, questions for the community:
   1) What is the best way to attach the test case ID to the test case?
       - We could add a new 'TestCaseID' keyword to Gherkin, and that wouldn't be specific to HPQC; it would support any test management framework.
       - What other choices are there? Is one of them clearly better?

I would just use a tag for this, but you could also use a comment. 

@test-case-id-123456

Scenario: Login with wrong password

  Given ...

or

#test-case-id-123456

Scenario: Login with wrong password

  Given ...

Either way, you'll probably need to write a custom formatter to grab this information for your purposes as the tests are run.

There's an alternative, which is the barely-documented 'scenario ID'. This is exposed in the JSON output from Cucumber[1], and uses the path to the file plus the name of the scenario to generate a readable unique ID for the scenario. It does depend on you giving all scenarios unique names which isn't currently enforced but probably will be in future versions.

Here's some details:

https://github.com/cucumber/gherkin/pull/278

[1] https://github.com/cucumber/gherkin/blob/master/features/json_formatter.feature#L119

   2) How do we extend Cucumber to call the HPQC API and record the test results?

There are two ways. You can use an After hook to get some information about the scenario, but you may be better looking at a custom formatter. Both Ruby and JVM versions expose an API that fires events out as your tests run.

I want to create something that will be useful to the broad community rather than something that meets my needs and only my needs. Who can point me in the right direction?
 

--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

cheers,

Matt

--

tel. +44 (0)20 8720 6842

http://mattwynne.net || https://twitter.com/mattwynne || http://the-cucumber-book.com || http://bddkickstart.com || http://www.relishapp.com

[Leslie Brooks]

Vincent, have you used the ALM Synchronizer SDK? It requires a signed license agreement, where the OTA API is freely available to anyone who has an HPQC/ALM license. We have custom test tools that use OTA to update test case execution status and results in HPQC; that isn't hard. However, we haven't done bulk export of test cases from HPQC; that may be easier with the synchronizer SDK.

[Leslie Brooks]

On Monday, January 20, 2014 10:36:37 AM UTC-5, Leslie Brooks wrote:

Vincent, have you used the ALM Synchronizer SDK? It requires a signed license agreement, where the OTA API is freely available to anyone who has an HPQC/ALM license. We have custom test tools that use OTA to update test case execution status and results in HPQC; that isn't hard. However, we haven't done bulk export of test cases from HPQC; that may be easier with the synchronizer SDK.

One knowledgeable source told me that with QC 11 (and ALM 11) there is a new REST API that obsoletes OTA. He recommended writing an external app that uses this API to show the Test Set tree from HPQC. The user selects the correct Test Set (collection of Test Cases (HPQC terminology) or Test Scenarios (BDD terminology)) and clicks 'Run'. The app then reads the test case descriptions (the BDD scenarios) and the test case ID and formats them for Cucumber. Then it runs Cucumber, and Cucumber uses the test case IDs (embedded in a tag) to update QC with the execution results.

If we really want to get fancy we can trace the QA test cases back to their parent requirements; all test cases that go to the same requirement would be written to one Feature file, and all test cases that trace back to a different requirement would be written to a different Feature file. (This might assume that each test case maps to only one requirement, but QC allows one-to-many.)

[Leslie Brooks]

Useful info Matt; thank you. I will investigate these approaches.

[Terry Pescosolido]

Hi, Leslie.  Are you still pursuing this?  We're pondering this very same integration.  How well is it working for you?

Terry