Robot framework and HP Quality Center Integration

[Pod]

Hi all,

Does anyone has an experience on integrating RF with HP QC from triggering test execution till test result collecting similar to the way QC + QTP works? 

I'm looking for the way to use RF with QC but not sure if QC would have some API available for 3rd party test tool to use. My wish is to use QC as test management tool and RF as test tool. If QC does not support, any suggestion on other test management tool?

Thank you in advance,

Pod

[Bas Vodde]

Hi Pod,

Why would you want a test management system such as QC?

I've seen QC used a lot but I failed to understand the benefits it brings…

Bas

> --
> You received this message because you are subscribed to the Google Groups "robotframework-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-d...@googlegroups.com.
> To post to this group, send email to robotframe...@googlegroups.com.
> Visit this group at http://groups.google.com/group/robotframework-devel?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Pod]

Hi Bas,

I'm not sure if you have experience about QC/QTP solution before. What I am trying to achieve is to have a centralize test management tool and QC is a tool my team has been using for a long time so all test cases are already stored in there.

Pod

On Thursday, April 4, 2013 4:38:11 PM UTC+7, Bas Vodde wrote:

Hi Pod,

Why would you want a test management system such as QC?

I've seen QC used a lot but I failed to understand the benefits it brings…

Bas

On 4 Apr, 2013, at 5:17 PM, Pod <saroc...@gmail.com> wrote:

> Hi all,
>
> Does anyone has an experience on integrating RF with HP QC from triggering test execution till test result collecting similar to the way QC + QTP works?
>
> I'm looking for the way to use RF with QC but not sure if QC would have some API available for 3rd party test tool to use. My wish is to use QC as test management tool and RF as test tool. If QC does not support, any suggestion on other test management tool?
>
> Thank you in advance,
> Pod
>
> --
> You received this message because you are subscribed to the Google Groups "robotframework-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-devel+unsub...@googlegroups.com.

[Lieping Xie]

Hi,

Not sure about QC, what we are using is TestLink (http://testlink.sourceforge.net/docs/testLink.php) and it also provides XMLRPC API (http://www.jetmore.org/john/blog/2012/02/testlink-1-9-3-api-documentation/) for external client.

So basically you can trigger RF tests to be run and parse the xml file and store the result back to TestLink or even integrate with Jenkins.

Best regards,

Xie Lieping, Frank

--

You received this message because you are subscribed to the Google Groups "robotframework-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-d...@googlegroups.com.

To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-devel?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Best regards,
Xie Lieping, Frank

[Jeremy Johnson]

You basically have 2 options here, depending on which route you want to go.

1.  QC supports what is called a VAPI-XP Test, which is a custom test type that can be written in python!  So you would basically create a VAPI-XP test that called RF through python and passed in the names of all the RF scripts.  This is the "old school" method as the capabilities has been around since before QC was QC.

2.  The second option is to use the QC API, the latest version of QC has a REST API, but most of what you need isn't avaliable in the REST API.  Rather take the standard COM API and use winPython to create a python wrapper for it.  And then create a custom script that runs your RF Scripts and uploads the results to QC.  If you wanted that script could be in a library file (stored in QC) and you could create a bunch of test cases in QC that load that library, and execute an RF script.  In such a way you would have the end-to-end integration.

Neither way is perfect, both are a lot of work to setup and a bit janky.   The better option is to ditch QC and use something a bit more open like JiRA or Trac, but I realize that isn't possible for a lot of companies. If you do want to go down this road I would seriously consider that for a team scripting in RF QC provides no benefits whatsoever.  So you might have your scripting team just use RF and a version control system, but have the reports uploaded to QC so "others" can see the status of the execution.  That way you minimize the amount of QC integration that you need.  Of course only you can decided what is right for your company.  Anyways good luck.  Ping me if you have more specific questions.

Cheers,

Jeremy

[Pod]

Hi Frank,

Thank you very much for suggestion. I am also looking at Testlink and XMLRPC as an alternative. 

One thing I notice is I still cannot find the way (or workaround) to initiate test execution to RF with series of test cases at once. It seems to me that I have to execute one by one using "execute and save results" button. By the way, I'm new to Testlink so I don't know if Testlink can execute batch of test cases but I don't know how to do it or actually Testlink does not support it.

Do you happen to know how would I do that?

Pod

On Thursday, April 4, 2013 11:01:12 PM UTC+7, Lieping Xie wrote:

Hi,

Not sure about QC, what we are using is TestLink (http://testlink.sourceforge.net/docs/testLink.php) and it also provides XMLRPC API (http://www.jetmore.org/john/blog/2012/02/testlink-1-9-3-api-documentation/) for external client.

So basically you can trigger RF tests to be run and parse the xml file and store the result back to TestLink or even integrate with Jenkins.

Best regards,

Xie Lieping, Frank

On Thu, Apr 4, 2013 at 5:17 PM, Pod <saroc...@gmail.com> wrote:

Hi all,

Does anyone has an experience on integrating RF with HP QC from triggering test execution till test result collecting similar to the way QC + QTP works? 

I'm looking for the way to use RF with QC but not sure if QC would have some API available for 3rd party test tool to use. My wish is to use QC as test management tool and RF as test tool. If QC does not support, any suggestion on other test management tool?

Thank you in advance,

Pod

--
You received this message because you are subscribed to the Google Groups "robotframework-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-devel+unsub...@googlegroups.com.

To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-devel?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Lieping Xie]

Hi,

For TestLink/RF/Jenkins integration it requires some extra effort

- first of all, we need the mapping between TestLink test case and RF case

  - we can use customized field in TestLink to map TestLink case with RF case by case name

  - or we can use tags for RF case which point to corresponding TestLink case id which is unique

- second of all, create a plan and add TestLink cases into that plan

- next step is to use TestLink XMLRPC API to access that plan to retrieve all the cases information (either customized field value or id) and run the RF with suite/test name (-s or -t option) or tags (-i option), so that all the required tests can be run within one command.

- last is to parse the xml output file generated by RF and use XMLRPC API again to update the case status in TestLink

Hope it clears and helps

Best regards,

Xie Lieping, Frank

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-d...@googlegroups.com.

To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-devel?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Pod]

Hi Jeremy,

Thank you so much for such a great detail, opinion, and suggestion you gave. Really appreciate.

Let me discuss with my team based on your information to see what they think. 

Thanks,

Pod

[Bas Vodde]

Hi Pod,

In most cases where I've seen RF adoption, QC just wasn't useful anymore. That's why I'm not really understanding why you would want a link. It sounds like the best idea is to completely ditch QC :)

What benefit does QC provide over just RF cases in svn or git?

THanks,

Bas

On 4 Apr, 2013, at 6:05 PM, Pod <saroc...@gmail.com> wrote:

> Hi Bas,
>
> I'm not sure if you have experience about QC/QTP solution before. What I am trying to achieve is to have a centralize test management tool and QC is a tool my team has been using for a long time so all test cases are already stored in there.
>
> Pod
>
>
> On Thursday, April 4, 2013 4:38:11 PM UTC+7, Bas Vodde wrote:
>
> Hi Pod,
>
> Why would you want a test management system such as QC?
>
> I've seen QC used a lot but I failed to understand the benefits it brings…
>
> Bas
>
> On 4 Apr, 2013, at 5:17 PM, Pod <saroc...@gmail.com> wrote:
>
> > Hi all,
> >
> > Does anyone has an experience on integrating RF with HP QC from triggering test execution till test result collecting similar to the way QC + QTP works?
> >
> > I'm looking for the way to use RF with QC but not sure if QC would have some API available for 3rd party test tool to use. My wish is to use QC as test management tool and RF as test tool. If QC does not support, any suggestion on other test management tool?
> >
> > Thank you in advance,
> > Pod
> >
> > --
> > You received this message because you are subscribed to the Google Groups "robotframework-devel" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-d...@googlegroups.com.

> > To post to this group, send email to robotframe...@googlegroups.com.
> > Visit this group at http://groups.google.com/group/robotframework-devel?hl=en.
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
>

> --
> You received this message because you are subscribed to the Google Groups "robotframework-devel" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-d...@googlegroups.com.

[Vernon Crabtree]

This is an old post, but I keep coming back to it in various searches I do, so for anyone else wandering this way:

I have integrated Robot Framework into Quality Center.

Why?  Simple, Quality Center is our corporate Test Management tool (OK, not the sexiest tool out there and it has its annoyances, but it is widely used and covers a wide range of needs). 

Quality Center is not a test run tool.  HP would recommend Unified Test Facilitator for this.

We didn't want to buy UTF for >100 testers, so went with open-source Robot Framework instead.

All test definition data is stored in Quality Center - scripts, parameters, attachments etc.  Tests are activated from QC and results are posted back to QC with details attached to the test run.

The integration script is written in python.

I am curious to know how much interest there is for this.  There might be a possibility to open-source this integration if someone is willing to help maintain it.

[Ivan Mitev]

Hi Vernon,

I would be interested to learn more about your solution and examine your code. I have some experience with HPQC (mostly interacting with the OTA API with tests, test sets, defect etc.) and I also know RobotFramework well. If you post your code in github or bitbucket (or you can mail it to me), I would be interested to take a look. Not sure I would help with maintenance, but at the very least I would give you feedback (if you need).

Cheers

[Clayton Neal]

Hello.

Would be very keen on taking a look at how this was done
Am just in the process of proposing RF to where i work, and they use QC

Thanks.

[Mun Sei Loke]

Hi Vernon,

Would you mind to share how did you integrate RF into QC?

Thanks.

[vittal vicky]

Dear Vernon,

I think you can post the same in Github where the maintainers of robot framework would be interested in your approach and possibly they can maintain the scripts.

[JJ Ledoux]

Hey Vernon, 

Everyone is asking about your approach here, if you could share we would all really appreciate it. 

Thanks!

[Unnikrishnan V]

Hi Vernon,

I am thinking of integrating HPQC and RF for the same reasons you mentioned.

I would appreciate if you could share your approach and I extend any help from my part to maintain this integration in future.

Regards,

Unni

[Vernon Crabtree]

At work we have started a KanBan board.  We needed to put post-its with tasks on there.

My name disappeared under the post-its!  I have been literally buried by tasks.

However, have a brief moment to look at my e-mail and see lots of queries about my integration.

Indeed we have successfully made this integration.

It works OK, but we are not really satisfied with the resulting situation.

The issues I am aware of with making a direct integration from VAPI-XP-TEST:

  * Version dependencies between Python and Quality Center.

  * We used COM, but HP is moving towards REST.  At the time we started (QC version 11), OTA did not support all functionality we needed.

  * Running larger sets of tests is cumbersome. Robot Framework has a nice way of handling Suite setup for a whole suite - Quality Center is not so useful in this regard.

  * For integration with a Continuous Integration server, Quality Center is a hinderance.  It is not impossible (there are integrations with Jenkins for example), but it is also not always helpful.

  * It turns out that storing lots of configuration data in Quality Center is not the desired approach.  Quality Center is a Test management tool - the test execution and associated data needs to be elsewhere.

A better approach is to develop all tests and data management with Robot Framework directly.  

Run Robot Framework independently.

Use a proper version control system to store the test sets (Quality Center is not so good at version controlling sets of information).

Synchronise the results back into Quality Center.  I have not yet decided how to associate the test in Robot Framework with the test in Quality Center - maybe some clever use of Tags can be used?

For us, this would be the next phase of evolution of our solution.  However, we are in the middle of a major merger so all such developments are on-hold for now.

I would be keen to know other thoughts on this - maybe between us we can think of something even better.

On Thursday, 4 April 2013 11:17:08 UTC+2, Pod wrote:

[Unnikrishnan V]

Hi Vernon,

Can you please share the basic code for this, so that we can discuss on how to make it better.

Thanks,

Unni

[ombre42]

We developed a crude solution to use OTA to upload test results, using test tags to link to either test plan IDs or test lab IDs. We used this for some time before we switched to SpiraTest.

Before developing, we already had a similar solution for TestNG with an intermediary XML format. So for Robot Framework, just used XSLT to covert output.xml to XML in same format as TestNG listener was outputting.

We run Jenkins, thus %WORKSPACE% is defined. That is also why you see the usage of environmental variables to pass information along - its the easiest way to do it with Jenkins we found.

The link between RF tags and QC is difficult to maintain, and prone to human error.

I hope this might be some use to you.

Kevin

[Unnikrishnan V]

This is a very basic solution. We can explore from here. I used Active python 2.7.8.10.

If you paste this code in VAPI-XP script editor and import the test case to Test Lab, this will Run test cases from Test lab and update results.

   

    import os
    k = os.system('pybot Absolute\path\to\dirctory\containing\robot\data\source')
    TDOutput.Print(k)
    if not Debug:
       if k == 1:
          CurrentRun.Status = "Failed"
          CurrentTSTest.Status = "Failed"
       elif k == 0:
            CurrentRun.Status = "Passed"
            CurrentTSTest.Status = "Passed"

[clay...@gmail.com]

Here is a script we use at work for uploading our RF test results to QC: https://github.com/clayjard/qcri