retrieving tests list dynamically ?

[Sherif Fathalla]

Hi Team ,

is there an easy way to retrieve the list test case names that'll be run in a pybot call ?

meaning :

example:

pybot  --suite "Suite01" --test"*"  ~/testsuites    #--- > 50 test cases 

pybot  ~/testsuites     #--- > 100 test cases

after calling this and in the Suite setup , i'd like to get a @{List} of the test cases that'll be run , as i need to parse  the names it and update a test repository tool ( testrail) with the test cases list 

Any suggestions ?

Regards,

Sherif

[Kevin O.]

Is there a reason you need the list before execution starts?

With a listener you can see the tests as they run. With the output XML, you can get the results afterwards.

Kevin

[Sherif Fathalla]

Thanks Kevin ,

we are merging our automation test cases with the manual tests into on test repo using a tool ( testrail ) , basically in testrail we would create a testrun containing all the test cases to be executed , initially they are all in a "Untested" state , then after each test completes the results is posted ( passed/Failed/Blocked etc ) .

so when I run robot , it uses a testtrail library that we created that basically adds ALL the test cases in the suite using their testrail IDs ,  ( I had to hardcode that list manually  which is far from ideal ) then in the teardown of each test cases i check if the test case Passed or failed and i grab the case ID from ${Test_NAME} using regex and update the testrail testcase result. 

but now that is limiting me because if I run (1 test case for example  --test "1200"  )  it adds All the hardcoded test cases in the beginning and only one result is posted "1200" .

I could also add each test case at a time in the [Setup] however when the project managers view the testrun progress it doesn't give a proper indication of what will run/is running/current passing % etc

that's why I'd like to be able to get the list dynamically in the beginning > get the testcase IDs and then add them to run

[Kevin O.]

We never had the requirement of having any status updates until execution was completed. If your execution is many hours long, I could see why one might want to see progress. Have you considering using more parallel testing?

I'm not sure what the solution to your issue is. I would try using the TestSuiteBuilder API. Look at how TestDoc is implemented.

We have integrated with both Quality Center and SpiraTest. For both, we tag tests with tags that fit a particular format.

For example, ST_ID=TC020123, QC_ID=27192, and ST_Set=FreeShipping. This keeps metadata out of the test name. The downside is the statistics are bad unless you use the --tagstatexclude to ignore these tags.

One solution uses a listener to upload the test results at the end and the other uses post-processing outside of RF entirely (processes output XML). Both approaches seem to work, but using a listener is a little more user-friendly.

[Ss Raghavan]

One way to get all the test cases list to be retrieved is based on the following conditions
1) If there are no parameters , then use
From robot.libraries import TestData

Use TestDAta class to parse the test case and map it to the list/dict and update testrails

If there are parameters accordingly use getopts and parse the suite file accordingly

It's very easy I have used in my project. Trust me it worked superb that way

[Pekka Klärck]

2014-08-13 10:23 GMT+03:00 Sherif Fathalla <sherif....@gmail.com>:
> Hi Team ,
> is there an easy way to retrieve the list test case names that'll be run in
> a pybot call ?

Currently you can only get this programmatically using Robot's APIs to
parse test data or to analyze previous results (assuming no tests have
been added or removed). This is a functionality I'd like to see in
TestDoc one day, though. LibDoc can already print information on
console (try e.g. `python -m robot.libdoc Selenium2Library list` and
see `python -m robot.libdoc --help` for details), and similar features
in TestDoc would be handy.

Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

[Sarah McK]

We use a third party test mgmt tool too (Zephyr) and parse the output.xml every 3 minutes and update Zephyr test status with that.  It's not perfect, cause you cannot flip the status of tests that *should* have passed til the end, but it helps to broadcast failed tests during the run.

[Sherif Fathalla]

Thanks all for your answers , I'' try using the APIs per your advice and let you know how it goes. :)

[Bryan Oakley]

Do you need this information before running the test, or can it wait until after it has run? If it can wait, you can use the listener interface to save each test case name as it is run, and write it to a database, send it to a web service, write it to a file, or whatever you want. 

If you need ti before, you can use the --dryrun option which doesn't do a real execution, but the output.xml file can be parsed to get all the test cases and suites it would have run. See http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#dry-run for more information

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-u...@googlegroups.com.
To post to this group, send email to robotframe...@googlegroups.com.
Visit this group at http://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.