Jenkins & RobotFramework & TestLink

[alans09]

Hello it's third day now that i'm looking for solution to my problem. First of all let sum up my setup:

1) I have installed TestLink with enabled API

2) I have installed Jenkins and robotframework

I have configured jenkins, I can get data from testlink, execute robot framework like this:

pybot --variable BROWSER:chrome --variable URL:http://<hidden url> ./TestCases/$TESTLINK_TESTCASE_ROBOT;

where TESTLINK_TESTCASE_ROBOT ist ID of testcase.txt file

Problem is that I can't get working that uploading result into testlink.

I always get information about test-cases that are not run!

But I can see clearly that they are executed and without error.

I tried to create   'result.tap file' with content:

result.tap:

1..2

ok 1

ok 2

and set this file as include pattern in TAP file name   in   result seeking strategy.

In log there is a message that says that there is information about results found, but testlink testcases are not updated. 

Console output log:

Found 1 automated test cases in TestLink.
Sorting automated test cases by TestLink test plan execution order.
Executing single Build Steps.
Executing iterative Build Steps.
Merging build environment variables with data retrieved from TestLink.
[workspace] $ /bin/sh -xe /tmp/hudson8139571206764439847.sh
+ pybot --variable BROWSER:chrome --variable URL:<hidden> ./TestCases/01_LoginTest.txt
==============================================================================
01 LoginTest                                                                  
==============================================================================
Prihlas sa do aplikacie ako <hidden> s heslom <hidden>           | PASS |
------------------------------------------------------------------------------
01 LoginTest                                                          | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /var/lib/jenkins/jobs/FullRegression/workspace/output.xml
Log:     /var/lib/jenkins/jobs/FullRegression/workspace/log.html
Report:  /var/lib/jenkins/jobs/FullRegression/workspace/report.html
Looking for the test results of TestLink test cases.
Found 1 test result(s).
Xvfb stopping

Finished: SUCCESS

All i can get from this is message:

Total of 1 tests. Where 0 passed, 0 failed, 0 were blocked and 1 were not executed.

Can somebody help me with this? or this is just bad forum for this question?

Thank you in advance...

[Dezider Mesko]

Hi,

I don't have experience with setup you described, however first thing which came to my mind is, that you have issue with paths/working directories. Just from log i see that some script is running from /tmp and with relative path ./TestCases... as argument, and robot results are in workspace directory. I'm probably wrong, however just for case, check all your paths and try to use absolute paths or so...

d.

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/groups/opt_out.

[alans09]

Hi .. thank you for response. I found the problem.

First of all, you need  junit type of result.. So easiest way is to  add   -x junit.xml   to pybot.

Second of all, you have to check names and variables because:

I had testcase named 01_LoginTest.txt   but... when I used -x  parameter to pybot, it becomes  01 LoginTest  so there was problem with pairing to correct testcase in testlink.

After change of testcase name to LoginTest and of course also change of defined custom field value to LoginTest it all magicaly began to work.  Maybe this 'rename' of test file is bug in pybot. But after 3 days I finaly found it.

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-users+unsub...@googlegroups.com.

[Dezider Mesko]

Ok :-)

Replacing of underscores with spaces is kind of feature ;-)

http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.1#test-suite-name-and-documentation

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.

[Ankur Chhabra]

Hi 

I am trying to deploy exactly the same scenario. Can someone ple describe the required steps to setup Testlink with Jenkins and RobotFramework?

[Zied Bejaoui]

Hello,

I am in the phase of linking Robot Framework and testlink through Jenkins.

I managed to establish the connection however I am not sure how to proceed to feed testlink with my .robot files.

I can see that in your case you have created a variable and you are referring to a text file where you have probably listed your test case IDs could you please give me more details on how to do that?

[Nesat Ufuk]

Wow! Looking forward to hear about Jenkins + Testlink + Robot FW setup more detailed.
Can you please alaborate about how to go through this.

[Tomas Pekarovic]

Hello all....

 i'm not using this setup for a years. I will try to create some guide for this and I will post it. 

I found some basic info about this integration in my old emails.

This is how-to that I send to some testers out there. 

Prerequisites:

- Enabled automation support and generated UID in testlink

- Enabled testlink plugin in Jenkins and added UID into it

Steps to reproduce:

1) You have testcases  s1-t-1,  s1-t-2 ...  (in testlink)

2) Create custom field Java Class and add it to each testcase

    put this names(id-s) into  Java Class of each testcase.     this means, testcase identified by  s1-t-1  wil have this also in Java Class field. (I would recommend to rename this field, and use something without spaces/underscores...eg. Robot )

3) use exact the same ids as name of robot testcases...  testcase that is testing s1-t-1 will be  s1-t-1

   *** Test cases ***

   s1-t-1

        Log     Simple testcase

   s1-t-2

        Log     Simple testcases

4) use -x argument to produce xunit file ...  robot -x result.xml <suite/testname>   #just run tests whatever is your way to run it  it's importatnt to have xunit file created

5) in build step add Invoke TestLink  (in Jenkins)

6) use Result Seeking Strategy:  JUnit case name

   Include pattern set for that   result.xml

   Key Custom Field:   Java Class (Robot)

Maybe this will help... if not... I will try to make some time and write down detailed instructions how to deal with this integration.

--

You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.

Visit this group at https://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

[Vinicius Kwiecien Ruoso]

Hi Tomas,

This subject is quite interesting and I see you had success in the integration.

I could not understand where the Java part comes in the process, could
you elaborate on that?

Another question: you said you have not used this setup in years. Can
you tell us the direction you went from there?

Greetings,
Vinicius

[Tomas Pekarovic]

Hello guys,

As I promised. I made a first version of 'guide' to get integration working. It is not the best so feel free to comment and I will add another information or maybe fix some of them.

You can find it here: http://alans09.blogspot.com/2016/02/robot-framework-jenkins-testlink.html

Answer to Vinicious question:

- We are using Jira and Jenkins and TestLink for manual execution. We stopped to use TestLink for automatic script execution. We build our own dashboard that integrates all our Test Requests from Jira and results from jenkins (standard robot framework report). We have all in one place and its easier for us and for manual testers to get information from.

Tomas

[Ellendula Sainath]

Hi Tomas,

Thanks for providing below link.

Followed the steps as per the below provided link to integrate Robotframework with jenkins & Testlink

Testlink: 

1)Created and added two testcases as sip-001 & sip-002 added to test suite. Each testcase attached to Robot script.. Pls find attached screen shot for the same.

Note:- robot: sip-001

2) I don't see any issue with testlink.

Jenkins: 

1)We followed as per below provided link. Here we are facing issues with shell command which provided in link, As we are using windows batch command.

Using below bash command:.. Guide me if i am wrong.

DEL TREE run.robot || True

ECHO "*** Test Cases ***" > run.robot

ECHO "sip-001" >> run.robot

ECHO "   Log    sip 01" >> run.robot

ECHO "sip-002" >> run.robot

ECHO "   Log    sip 02" >> run.robot  

2) Added  pybot -x junit.xml run.robot

# this will run run.robot file and creates also junit.xml as junit result file.

3) Publish JUnit test result report given as "Test-Report.xml" But getting error with "no matching as Test-Report.xml".. Pls guide me for correct syntax to follow

After saving and run this got below error.

Building in workspace C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace

[workspace] $ cmd /c call C:\Windows\TEMP\hudson2927018926311170411.bat

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>DEL TREE run.robot   || True

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>ECHO "*** Test Cases ***"  1>run.robot 

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>ECHO "sip-001"  1>>run.robot 

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>ECHO "   Log    sip 01"  1>>run.robot 

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>ECHO "sip-002"  1>>run.robot 

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>ECHO "   Log    sip 02"    1>>run.robot 

C:\Program Files (x86)\Jenkins\jobs\Integration-test\workspace>exit 0 

Preparing TestLink client API.

Using TestLink URL: http://localhost:81/testlink/lib/api/xmlrpc/v1/xmlrpc.php

Found 2 automated test cases in TestLink.

Sorting automated test cases by TestLink test plan execution order.

Executing single Build Steps.

Executing iterative Build Steps.

Looking for the test results of TestLink test cases.

Looking for test results in JUnit test cases by its name.

FATAL: hudson.AbortException: No test report files were found. Configuration error?

hudson.plugins.testlink.result.ResultSeekerException: hudson.AbortException: No test report files were found. Configuration error?

        at hudson.plugins.testlink.result.JUnitCaseNameResultSeeker.seek(JUnitCaseNameResultSeeker.java:110)

        at hudson.plugins.testlink.TestLinkBuilder.perform(TestLinkBuilder.java:236)

        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)

        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)

        at hudson.model.Build$BuildExecution.build(Build.java:205)

        at hudson.model.Build$BuildExecution.doRun(Build.java:162)

        at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)

        at hudson.model.Run.execute(Run.java:1738)

        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)

        at hudson.model.ResourceController.execute(ResourceController.java:98)

        at hudson.model.Executor.run(Executor.java:410)

Caused by: hudson.AbortException: No test report files were found. Configuration error?

        at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:116)

        at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:92)

        at hudson.FilePath.act(FilePath.java:990)

        at hudson.FilePath.act(FilePath.java:968)

        at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:89)

        at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:43)

        at hudson.tasks.test.TestResultParser.parse(TestResultParser.java:145)

        at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:74)

        at hudson.plugins.testlink.result.JUnitCaseNameResultSeeker.seek(JUnitCaseNameResultSeeker.java:88)

        ... 10 more

ERROR: An error occured while trying to retrieve the test results: hudson.AbortException: No test report files were found. Configuration error?

Recording test results

ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?

Finished: FAILURE

Any suggestion would be very helpful

Regards,

Sainath

>> email to robotframework-users+unsub...@googlegroups.com.

>> To post to this group, send email to
>> robotframe...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/robotframework-users.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "robotframework-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to robotframework-users+unsub...@googlegroups.com.

[Hélio Guilherme]

terça-feira, 5 de Abril de 2016 às 14:35:03 UTC+1, Ellendula Sainath escreveu:

(...)

Preparing TestLink client API.

Using TestLink URL: http://localhost:81/testlink/lib/api/xmlrpc/v1/xmlrpc.php

Found 2 automated test cases in TestLink.

Sorting automated test cases by TestLink test plan execution order.

Executing single Build Steps.

 

I think you forgot to add  this step in Jenkins!

2) Added  pybot -x junit.xml run.robot

# this will run run.robot file and creates also junit.xml as junit result file.

If you look at the output in the tutorial, there is the Robot Framework output there.

[Tomas Pekarovic]

Hello,

If you added    pybot -x junit.xml run.robot   into build step then the final junit xml output is named   junit.xml.

From your post (and jenkins log) there is no 'pybot' build step execution. This means that no xml is created.

After that you are trying to publish xml named Test-Report.xml.

(Publish JUnit test result report given as "Test-Report.xml" But getting error with "no matching as Test-Report.xml".. Pls guide me for correct syntax to follow   < Quote from your post)

This means if you don't run that robot framework (pybot) execution step, you don't have report at all. Try to add pybot -x junit.xml run.robot as next step after 'Echo-ing' test.  After that in your workspace there should be file called junit.xml . This file you want to publish as result and use for testlink. (Not  Test-Report.xml mentioned above.  Or you can just   pybot -x Test-Report.xml run.robot )

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.

[Ellendula Sainath]

Hi,

Thanks for your response.

In Testlink i have created Testcase ID and my robot script is attached as attachment. Please find screen shot of the same. 

Now, In jenkins i could able to access testlink Testcases ID. But i Don't know how to execute Testlink attached Testcases.

In jenkins when i tried to build now.. Testcases ID are fetched from testlink but attached testcases are not executing, showing execution status as "Not executed". Please find attachment of jenkins output.

Please help me how to execute testlink attached TCs(Robot scripts) from jenkins. Any commands would be very helpful.

Regards,

Sainath

On Wednesday, April 6, 2016 at 11:28:55 AM UTC+5:30, Tomas Pekarovic wrote:

Hello,

If you added    pybot -x junit.xml run.robot   into build step then the final junit xml output is named   junit.xml.

From your post (and jenkins log) there is no 'pybot' build step execution. This means that no xml is created.

After that you are trying to publish xml named Test-Report.xml.

(Publish JUnit test result report given as "Test-Report.xml" But getting error with "no matching as Test-Report.xml".. Pls guide me for correct syntax to follow   < Quote from your post)

This means if you don't run that robot framework (pybot) execution step, you don't have report at all. Try to add pybot -x junit.xml run.robot as next step after 'Echo-ing' test.  After that in your workspace there should be file called junit.xml . This file you want to publish as result and use for testlink. (Not  Test-Report.xml mentioned above.  Or you can just   pybot -x Test-Report.xml run.robot )

On Tue, Apr 5, 2016 at 4:34 PM, Hélio Guilherme <heliox...@gmail.com> wrote:

terça-feira, 5 de Abril de 2016 às 14:35:03 UTC+1, Ellendula Sainath escreveu:

(...)

Preparing TestLink client API.

Using TestLink URL: http://localhost:81/testlink/lib/api/xmlrpc/v1/xmlrpc.php

Found 2 automated test cases in TestLink.

Sorting automated test cases by TestLink test plan execution order.

Executing single Build Steps.

 

I think you forgot to add  this step in Jenkins!

2) Added  pybot -x junit.xml run.robot

# this will run run.robot file and creates also junit.xml as junit result file.

If you look at the output in the tutorial, there is the Robot Framework output there.

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-users+unsub...@googlegroups.com.

[Kranti Kumar]

Hello Alans09,

The link you have provided to integrate Jenkins, Testlink and Robot is very useful(https://alans09.blogspot.in/2016/02/robot-framework-jenkins-testlink.html?showComment=1492040877930#c2769665950132006022).

We would like to add JIRA to this existing setup. Could you please guide me or provide me the step by step procedure if it is possible.

Thank you so much in advance.
Regards,
-kranti