Test Case Management?
Carl Dichter
We like the features of TestTrack TCM www.seapine.com/tttcm.html but I
hope it isn't going to be a lot of work to get it integrated with
RobotFramework.
--Carl
Pekka Klärck
> Is anyone using RobotFramework with a Test Case Management tool.
I believe most people simply use their preferred version control
system for storing the tests. There has been some discussion about a
web based management system but although some prototype code has been
written [1] this project hasn't yet moved much forward.
> We like the features of TestTrack TCM www.seapine.com/tttcm.html but I
> hope it isn't going to be a lot of work to get it integrated with
> RobotFramework.
I also hope integrating these tools isn't too complicated. Let us know
how everything works out and especially if RF could be enhanced
somehow to make such integration easier.
Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org
Carl Dichter
+ historical reporting capabilities (that perhaps Risto.py can do) with
trending per test-case and for the suite,
+ ability to record manual testcases (e.g., Mabot),
+ being able to estimate the execution time of a selection of (manual and
automated) tests,
+ and other features like ability to declare whether testcases are enabled
for automation (as the keywords become available).
+ and I want all of this to be easy for management to use without training
or multi-step installation.
If there is a way to do it all with FOSS software, then that would be
fantastic!
--Carl
Magnus
(http://www.teamst.org/), which we use to keep track of our manual
(legacy) test cases.
The idea is that results from RF runs can be automatically imported
into TestLink, and that test cases (in TestLink & RF) will link to
each other so that it's easy to jump between the two.
Unfortunately, the work has slipped down a bit in priority, so it
hasn't been completed yet, but I don't think it will be very difficult
since RF presents results in a simple xml-format.
I don't know anything about TestTrackTCM, so I don't think I can help
you there.
However, if we do finish our integration with TestLink, I can post the
steps somewhere, and maybe that could be of some help to you.
Speaking of that:
I think it would be good if there were a place where RF users could
share their experiences, and tips-and-tricks with other RF users.
Right now a lot of good information just sinks into the abyss of the
discussion forum here. Not saying that this isn't a good forum for
discussions, but it would be nice with a complement, like a user wiki,
or something. Maybe we can use the "pages" (http://groups.google.com/
group/robotframework-users/web) of this group for this? (I haven't
tried what can be done with them yet)
Pekka, and the rest of the RF team, do you have any opinions on this?
BRs
Magnus
On Jan 13, 4:54 am, "Carl Dichter" <carl.dich...@gmail.com> wrote:
> Test Case Management can me a lot of things. I'm talking about:
>
> + historical reporting capabilities (that perhaps Risto.py can do) with
> trending per test-case and for the suite,
> + ability to record manual testcases (e.g., Mabot),
> + being able to estimate the execution time of a selection of (manual and
> automated) tests,
> + and other features like ability to declare whether testcases are enabled
> for automation (as the keywords become available).
> + and I want all of this to be easy for management to use without training
> or multi-step installation.
>
> If there is a way to do it all with FOSS software, then that would be
> fantastic!
>
> --Carl
>
> -----Original Message-----
> From: Pekka Klärck [mailto:pekka.kla...@gmail.com]
> Sent: Tuesday, January 12, 2010 4:39 PM
> To: carl.dich...@gmail.com
>
> Cc: robotframework-users
> Subject: Re: Test Case Management?
>
> 2010/1/12 Carl Dichter <carl.dich...@gmail.com>:
> > Is anyone using RobotFramework with a Test Case Management tool.
>
> I believe most people simply use their preferred version control
> system for storing the tests. There has been some discussion about a
> web based management system but although some prototype code has been
> written [1] this project hasn't yet moved much forward.
>
> [1]http://groups.google.com/group/robotframework-users/browse_thread/thr...
> 376bd0e370168/ab7b66b649bff856?#ab7b66b649bff856
>
> > We like the features of TestTrack TCMwww.seapine.com/tttcm.htmlbut I
Juha Rantanen
> Speaking of that:
> I think it would be good if there were a place where RF users could
> share their experiences, and tips-and-tricks with other RF users.
> Right now a lot of good information just sinks into the abyss of the
> discussion forum here. Not saying that this isn't a good forum for
> discussions, but it would be nice with a complement, like a user wiki,
> or something. Maybe we can use the "pages" (http://groups.google.com/
> group/robotframework-users/web) of this group for this? (I haven't
> tried what can be done with them yet)
> Pekka, and the rest of the RF team, do you have any opinions on this?
>
> BRs
> Magnus
>
Hi Magnus,
I think that is great idea. We have to investigate which Wiki we could
use for that purpose. The problem with Google code Wiki is that it is
not easy to allow access for anonymous changes and that is what Wiki
is all about.
Br,
Juha
Magnus Smedberg
Have you tried out the "Pages" option that is built in with google groups? I haven't used it, though it seems ok, but it does seem to lack some of the advantages of a wiki. However, using it would have the advantage of proximity, it's located on the same page (more or less) as the discussions-forum.
Otherwise, are there any good web-sites that offers free hosted wikis?
BRs
Magnus
Carl Dichter
BRs
Magnus
> + with
> trending per test-case and for the suite,
> + ability to record manual testcases (e.g., Mabot), being able to
> + estimate the execution time of a selection of (manual and
> automated) tests,
> + and other features like ability to declare whether testcases are
> + enabled
> for automation (as the keywords become available).
> + and I want all of this to be easy for management to use without
> + training
Chris Prinos
looking to integrate it. TestTrack has a fairly complete SOAP API,
though I've had a hard time getting it to work with any SOAP client
implementations other then .NET (so I'm using IronPython to talk to
it). There's a lot of possible ways to integrate the two, our initial
approach is going to be something simple:
1. In Robot, test case names start with TCnnnn: where nnnn is the
number of a TestCase record in TestTrack. I played with the idea of
sourcing the robot test definition in TestTrack, but I think it is too
cumbersome to try to do that given the limitations of the editing
tools in TestTrack.
2. There will be a separate "results publisher" tool that takes the
output.xml from a robot run, and for each test with a TCnnnn in the
name, it will post the result to TestTrack. It does this by searching
for TestRun records that are associated with the TestCase number in
question. By keeping the publishing as a separate step you can have
TestTrack reflect the run results of only the robot runs that you are
interested in posting.
Sorry I don't have any code to share, some quick proof of concept
scripts did not take that long (wrestling with the TT SOAP interface
was the biggest obstacle), and it looked pretty straightforward to
have TT start showing robot results.
Chris
On Jan 12, 5:41 pm, Carl Dichter <carl.dich...@gmail.com> wrote:
> Is anyone using RobotFramework with a Test Case Management tool.
>
> We like the features of TestTrack TCMwww.seapine.com/tttcm.htmlbut I
Pekka Klärck
>
> 1. In Robot, test case names start with TCnnnn: where nnnn is the
> number of a TestCase record in TestTrack.
Why not to use tags for this purpose instead?
Cheers,
.peke
Chris Prinos
I'm trying to remember, I made that choice a while ago... I think it
was because:
- our TestTrack test cases didn't have 'short names', only a longer
description, so I figured I might as well use the test case number in
the robot test name since I wanted something short and unique.
Sometimes the names have an additional short description the form:
TCnnn: <short description>.
- It would have required adding a [tags] line to every test, whereas
the majority would not otherwise need test-specific tags (most of my
other tags are done at the suite level)
- if tags were used, you'd be forced to always use --tagstatcombine or
--tagstatexclude because every test would create a unique tag.
I can still pick and choose tests to run with -t tcnnnn* if needed, so
overall I don't think I really lost anything by not going with tags in
this case. Tags are still used for 'performance', 'smoke', component
areas, etc because then tags apply to groups of tests.
Chris
On Jan 14, 6:53 pm, Pekka Klärck <pekka.kla...@gmail.com> wrote:
> 2010/1/14 Chris Prinos <chrispri...@gmail.com>:
kri...@gmail.com
I've just downloaded Testtlink 1.9beta3, and it could be also be
useful for me to link it with robotframework.
I will have a look at it as a background activity for now.
The idea described by Magnus is what I had more or less in mind.
Starting to use testlink for manual testcases, and report the results
from the robotframework in testlink so 1 report can be generated. And
everything becomes easy accessible for other people. (management/
developers)
Kris
Radek
defined all test cases in TestLink. The ones that are automated in
robot are marked as 'Automatic' and assigned to a separate user. In
RobotFramework I prefix the test case name with the Test Link external
test case number (for example 'SLF-100 Test something'). I wrote a
simple script that takes he output.xml file are transforms it into a
file I can import into TestLink. This works until I figure out how to
use the TestLink web services to import results at runtime. Here is
the simple Python script I put together. You'll need lxml module,
replace 'SLF-' with your test case prefix, and assign the different
username to e_tester.text variable.
#!/usr/bin/python
import sys
import re
from datetime import datetime
from lxml import etree
out = etree.Element('results')
x = etree.parse(sys.argv[1])
tests = x.xpath('//suite/test')
for test in tests:
n = test.attrib['name']
match = re.match(r'^(SLF-\d*)\s.', n)
if match:
n = match.group(1)
status = test.find('status')
s = status.attrib['status']
ts = status.attrib['starttime']
ts = datetime.strptime(ts, '%Y%m%d %H:%M:%S.%f')
ts = ts.strftime('%Y-%m-%d %H:%M:%S')
t = status.text
e_tc = etree.Element('testcase', external_id=n)
e_tester = etree.Element('tester')
e_tester.text = 'robot'
e_tc.append(e_tester)
e_ts = etree.Element('timestamp')
e_ts.text = ts
e_tc.append(e_ts)
e_result = etree.Element('result')
e_result.text = s[0].lower()
e_tc.append(e_result)
if t:
e_notes = etree.Element('notes')
e_notes.text = t
e_tc.append(e_notes)
out.append(e_tc)
print(etree.tostring(out, pretty_print=True))
--
Radek
Magnus
I promised Pekka to make a page on TestLink integration here on the
user-group. Still haven't done it... :-(
Well, it's never too late, and now there seems to be some new ideas
so, maybe now is a good time.
What we've done in our TestLink integration is quite similar to
Radek's solution, with some differences. I'm using xslt to make the
transformation, and I'm putting the TestLink ids in tags instead.
For each RF test case, we create a corresponding TestLink test case,
and then tag the RF test case with "testlink_PRE-1234" where PRE-1234
is the TestLink id, including prefix.
We've also created a specific user in TestLink (called
"robotframework") for tests run from RF.
Then after each test run we just run the output.xml through an xslt
parser (Saxon in our case) along with the stylesheet below.
Unfortunately, the last step is still manual, which is to open
TestLink, go to Execute, click on a test case, and chose to import xml-
file.
I, too, want to move towards using TestLink's web services, but this
hasn't happened yet either. I've also gotten some good tips from Pekka
on how to use the internal RF API to post-process the results. Maybe
I'll get around to do the long planned improvements soon...
Cheers,
Magnus
==================================================================
<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<results>
<xsl:for-each select="//test">
<xsl:if test="starts-with(tags/tag[starts-
with(text(),'testlink')],'testlink')">
<testcase external_id="{substring-after(tags/tag[starts-
with(text(),'testlink')],'testlink_')}">
<tester>robotframework</tester>
<xsl:choose>
<xsl:when test="status/@status ='PASS'">
<result>p</result>
<notes>Executed automatically using Robot Framework.</notes>
</xsl:when>
<xsl:otherwise>
<result>f</result>
<notes><xsl:value-of select="status"/> - Executed automatically
using Robot Framework.</notes>
</xsl:otherwise>
</xsl:choose>
</testcase>
</xsl:if>
</xsl:for-each>
</results>
</xsl:template>
</xsl:stylesheet>
==================================================================
cdekter
webservices to update the test runs in SpiraTest. We've been using it
successfully for months now...
Radek
xmlrpc. It's trivial to implement a listener that will upload the
result at the end of every test case. However, I have few problems
here.
I have to somehow pass the browser name and build number to the
listener so that the test execution is saved under proper build-
environment combination. I have the browser name in test cases as a
robot variable but listeners can't access those. I'm stuck here.
Magnus, can you say something more about how you were planning to post-
process the results?
Or more generic question: can listeners access robot variables?
--
Radek
On Apr 8, 3:01 am, Magnus <magnus.smedb...@gmail.com> wrote:
> I, too, want to move towards using TestLink's web services, but this
> hasn't happened yet either. I've also gotten some good tips from Pekka
> on how to use the internal RF API to post-process the results. Maybe
> I'll get around to do the long planned improvements soon...
>
> Cheers,
> Magnus
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To post to this group, send email to robotframe...@googlegroups.com.
To unsubscribe from this group, send email to robotframework-u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/robotframework-users?hl=en.
Pekka Klärck
The discussion related to integrating Robot Framework with different
test management tools is really interesting. If it's possible to
create generic solutions, it would be great to have them available
somewhere with adequate documentation. It's possible to add them into
the main project as new supporting tools, or alternatively we can
start a new project to collect similar smallish tools.
2010/4/8 Radek <rade...@gmail.com>:
> xmlrpc. It's trivial to implement a listener that will upload the
> result at the end of every test case. However, I have few problems
> here.
>
> I have to somehow pass the browser name and build number to the
> listener so that the test execution is saved under proper build-
> environment combination. I have the browser name in test cases as a
> robot variable but listeners can't access those. I'm stuck here.
dynamically, you can either create tags using variables or use
--settag option or Set Tags keyword. I would similarly use tags to
specify test case IDs if you need them.
If you really want to use variables, you can access them via the
BuiltIn library keywords like this:
from robot.libraries.BuiltIn import BuiltIn
vars = BuiltIn().get_variables() # requires RF 2.1.3
var = BuiltIn().replace_variables('${MY VAR}')
> Magnus, can you say something more about how you were planning to post-
> process the results?
executed tests using RF's internal modules. Although that would work,
I actually think the approach used in your (=Radek) script to read the
information from the XML directly using normal XML modules is better
in this case. That approach is definitely faster, and as your script
demonstrated it's not so complicated either. I would just use the
standard ElementTree module (at least as a fall-back) to avoid extra
dependencies.
Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org
Magnus
Yes Pekka, I was referring to your tip about using TestSuite from
robot.output
Like:
from robot.output import TestSuite
suite = TestSuite('output.xml')
for test in suite.tests:
# transfer test case results over xmlrpc to TestLink
You're right, this will probably be slower, but the code is very
readable. (as opposed to my xslt)
To me, post-processing seems a lot simpler, rather then having
listeners at the end of each test case.
Radek, I assume that the build number, and browser name is information
that you pass into the RF test execution by command-line variables, or
variable files, right?
Then, why don't you just pass the same values into the post-
processing? (if you trigger it from your runner script) I think that
beats having it in tags, actually, unless you want it to be clearly
visible in the RF report as well.
I agree that this discussion is really interesting, and I would really
like us to conclude the things said here (when we're done) and put it
somewhere more easily reachable (if not as a standard part of RF).
Tell you what, I'll start by creating a page here at the user-group,
and then we'll see where it goes from there. I don't have much time to
spend on this, unfortunately, so I'll need help in getting the right
things on the page.
Cheers,
Magnus
On Apr 8, 11:23 pm, Pekka Klärck <pekka.kla...@gmail.com> wrote:
> Hi all,
>
> The discussion related to integrating Robot Framework with different
> test management tools is really interesting. If it's possible to
> create generic solutions, it would be great to have them available
> somewhere with adequate documentation. It's possible to add them into
> the main project as new supporting tools, or alternatively we can
> start a new project to collect similar smallish tools.
>
> 2010/4/8 Radek <radek...@gmail.com>:
Magnus
I wrote a start for a page here
http://groups.google.com/group/robotframework-users/web/integrating-rf-with-tcm-tools
You should also be able to find it by clicking on pages on the left
hand side menu of any discussion (unless you're reading this as an
email of course).
Please help me in filling this page with interesting and useful tips
on how to integrate RF with TCMs.
Cheers,
Magnus
Radek
> Radek, I assume that the build number, and browser name is information
> that you pass into the RF test execution by command-line variables, or
> variable files, right?
> Then, why don't you just pass the same values into the post-
> processing? (if you trigger it from your runner script) I think that
> beats having it in tags, actually, unless you want it to be clearly
> visible in the RF report as well.
Yes, I currently pass the browser name by command-line variable and I
planned the same for the build number.
I wanted to have the results in TestLink as soon as possible to be
able to see track them. That's why I picked the listener but it can be
more error-prone. I might move to post-processing because the import
might be repeated in case some errors occurred.
I'm having trouble with TestLink's XMLRPC in 1.9beta - it's buggy and
does not work at the moment.
--
Radek
ambi
integration :) - or let's say general Test Case Management.
I have personally tried the TestLink API. There is already a sample
code in python. I have tested it on TestLink version 1.8.5 and on
1.9.4 beta.
What I miss is how to actually use the ${TEST_STATUS} or "Run Keyword
If Test Failed" in every test case teardown and accordingly trigger
the TestLink API script to update the status of the test case as
"pass" or "fail".
Probably with "http://robotframework.googlecode.com/svn/tags/
robotframework-2.1.3/doc/libraries/BuiltIn.html#Run Keyword If"
For every test case in RF you would also need to pass the test case ID
which is the actual test case ID from TestLink.
e.g. of the TestLink API Client
def reportTCResult(self, tcid, tpid, buildid, status, notes):
data = {"devKey":self.devKey, "testcaseid":tcid,
"testplanid":tpid, "buildid":buildid, "status":status, "notes":notes}
return self.server.tl.reportTCResult(data)
Result = client.reportTCResult(1111, 2222, 33, "p", "executed")
So, what I do is - create the buildid according to the timestamp when
RF runs and pass it to reportTCResult. testcaseid will be passed
explicitly for every testcase and therefore one needs to know the
internal id for the test case from TestLink, the same for testplanid.
You could also maintain the IDs are variables in a resource file which
is used specifically for Testlink stuff.
Note that the above code is written for TestLink 1.8.5 which is why
the platformid is not present. Testlink version 1.9.4 requires
platformid.
I will play around a bit more this weekend :)
On Apr 9, 10:08 am, Magnus <magnus.smedb...@gmail.com> wrote:
> Ok, the ball is now in motion...
> I wrote a start for a page herehttp://groups.google.com/group/robotframework-users/web/integrating-r...
Pekka Klärck
> Yes Pekka, I was referring to your tip about using TestSuite from
> robot.output
> Like:
> from robot.output import TestSuite
> suite = TestSuite('output.xml')
> for test in suite.tests:
> # transfer test case results over xmlrpc to TestLink
>
> You're right, this will probably be slower, but the code is very
> readable. (as opposed to my xslt)
XSLT is pretty horrible language for anything non-trivial. The code
you got when using good XML parser such as ElementTree or lxml is much
better.
Magnus:
> I wrote a start for a page here
> http://groups.google.com/group/robotframework-users/web/integrating-rf-with-tcm-tools
> You should also be able to find it by clicking on pages on the left
> hand side menu of any discussion (unless you're reading this as an
> email of course).
Cool! This will also give practical knowledge about the usefulness of
Pages in Google Groups in general.
ambi:
> What I miss is how to actually use the ${TEST_STATUS} or "Run Keyword
> If Test Failed" in every test case teardown and accordingly trigger
> the TestLink API script to update the status of the test case as
> "pass" or "fail".
> Probably with "http://robotframework.googlecode.com/svn/tags/
> robotframework-2.1.3/doc/libraries/BuiltIn.html#Run Keyword If"
If you want to do updates at run time, it's probably better to use the
listener interface:
http://robotframework.googlecode.com/svn/tags/robotframework-2.1.3/doc/userguide/RobotFrameworkUserGuide.html#using-listener-interface
About using tags for IDs:
One reason why it might be a good idea to this approach is that you
can create links from tags to external systems:
http://robotframework.googlecode.com/svn/tags/robotframework-2.1.3/doc/userguide/RobotFrameworkUserGuide.html#creating-links-from-tag-names
marki
I'd be very interested to learn more about how you are using RF with
SpiraTeam. What exactly does it do for you? Are you storing test cases
in SpiraTeam, or just using SpiraTeam to log results?
Thanks,
Mark
cdekter
We don't use SpiraTeam but the simpler SpiraTest. Although since
SpiraTeam contains the features of SpiraTest it should work the same.
We are using the SOAP API provided in SpiraTest to add "runs" to
existing test cases, which includes pass/fail status and detailed
failure information.
Regards,
Chris
krisodb
arrived.
I was having a look at http://code.google.com/p/gallio-testlink-adapter/source/browse/#svn/trunk/TestLinkAPI
My intention was to keep the testcase names in RF the same as in
TestLink. Since it possible to retrieve the ID's based on the
testcase name I don't need anything special in RF. So no extra tags.
That's the way I learned it when I started to work, to format testcase
names in a specific way: x_y_z_01 where x,y,z are (sub)domain names
(you can have more/less levels if needed). That way have got a folder
structure (suite) like \x\y\z. All this comes back in RF and
testlink.
I will see that I can add some more info on this and maybe create some
seperate page incase you want to use ironpython instead of python to
use with RF.
But it will be for the coming months ... still some other libraries to
write to interface with my test object before I even need to update it
in testlink ...
Initially I was thinking about post processing, but on the fly update
might be more useful to monitor a long run ...
ambi
I have made a first working code for testlink to run with robot
framework. The code may be very primitive for most of you. I am not an
advanced python developer, which is why it took some time to get
around this. But, it was fun :)
Suggestions and comments are always welcome.
So here's what I did - I built the testlink api client using the
sample provided by testlink and extended it to my needs. I wrote a
config file to maintain the configuration seperate from robot. I also
extended run.py and I have a sample template for you guys to see how
it works.
1. TestLinkAPIclient.py
2. testconfig.ini
3. run.py
4. Sample template with an exclusive "testlink" resource file to
handle all testlink related keywords.
1. TestLinkAPIClient.py
"devKey", "testplanID", "buildNotes" are defined in the testconfig.ini
"buildName" is generated according to the timestamp.
"createBuild" creates the build using the buildName (which is the
timestamp) and returns the buildID. I assign this to a variable in RF
and use it globally.
"getTestCaseIDFromTestName" basically takes the test case name from RF
and evaluates it. I have a scheme to name my test cases in RF (ID_xxx-
This Is My TestCase) where xxx is the actual ID of the test case in
testlink. This ID is used in "reportTCResult"
2. testconfig.ini
A small configuration file (i thought i'll play a little more with
python) to pass values to devKey, testplanID and buildNotes as
mentioned above. I parse this config file with a small code written in
run.py.
All these variables are also initialized in the "testlink" resource
file in RF. See template "TestLink_Resource.txt"
I also have a switch functionality which tell RF to use testlink API
or not (TL_REPORT = TRUE or FALSE)
3. run.py
You may see some custom code there used for my own purposes, like
creating reports directories and sub directories in that.
Important is the read_cfg(). I parse the config.ini file and append
the parameters to DEFAULT_ARGS
In __main__ in just output all the arguments I have used with pybot.
4. The sample template
It consists of 2 Test Suites:
01 Prerequisites
02 TestSuite
and one resource file:
TestLink_Resource.txt
01 Prerequisites has a keyword to create a build. So a new build is
created at the start of RF, the buildID is returned and used globally.
02 TestSuite has 2 sample test cases with the test case naming scheme
I mentioned earlier.
Each of the test cases has a setup and teardown. For the setup I have
a keyword "Use TestLink" and teardown "Report To TestLink". These
keywords basically checked if TL_REPORT from the config file is TRUE
or FALSE. If TRUE then "Evaluate TestCase ID" is run as a setup where
the test name is evaluate and the testcase ID is returned. On teardown
"TestCase Result" is run where in if a ${TEST_STATUS} is PASS then
"TestCase Pass" is run and if FAIL then "TestCase Fail" is run.
TestCase Fail and TestCase Pass call "reportTCResult" from
TestLinkAPIClient and pass "testcaseID", "buildID", "status" and $
{TEST_MESSAGE}
I have tested this on TestLink 1.8.4
Files are in the "Files" section in this group - TestLink.zip
Eagerly waiting for feedback.
Cheers
Ambi
Dragonfyre13
other automation stuff (creating an entire platform with the sole
purpose of allowing RF to drive telephony systems) a couple months
ago, and haven't been able to dive back in yet. As I'm finished with
the project, I think I'll tackle this next.
I'm thinking with the below (Thanks ambi!), my life just got a whole
lot easier. My end goal is:
- Storage of RF cases in testlink as upload in root of suite tree
- Pre execution pulls down cases to execute from testlink
- Post execution submits results back to testlink via XMLRPC
Pretty sure the last one is satisfied below, at least for the most
part, although I'll likely run some cleanup for our internal use
(contributing that back when it's finished of course). Here's what
I've got planned:
- Modify the script so it uses the most recent build, instead of
creating a new one.
- Use a tag for Testlink ID (I like the "testlink_" prefix idea)
- Change the format of the config file (INI doesn't play nice with
non-windows) and add in more config options.
- Add significantly more logic surrounding failures and failure
reasons, making sure to upload descriptive notes to a failure.
Any other feature requests/comments while I'm at it?
ambi
Cheers for your feedback. I was beginning to think that my code is so
premature that no one bothered :)
I do have some more features on my list and I constantly try to
improve the existing API.
Shall we maintain a feature list somewhere . . "Issues" section as
"Enhancements" and have some sort of a formal process ?
Should we first formalize the list before before they're entered and
then link them or list them back to the "Robot Framework integration
with Test Case Management tools" page, so everyone sees in one glance
the features which are going to be implemented.
Let me know how I can be of any help.
Cheers
Ambi
Pekka Klärck
> Cheers for your feedback. I was beginning to think that my code is so
> premature that no one bothered :)
> I do have some more features on my list and I constantly try to
> improve the existing API.
>
> Shall we maintain a feature list somewhere . . "Issues" section as
> "Enhancements" and have some sort of a formal process ?
> Should we first formalize the list before before they're entered and
> then link them or list them back to the "Robot Framework integration
> with Test Case Management tools" page, so everyone sees in one glance
> the features which are going to be implemented.
or elsewhere? You already have some code and issues to list so having
version control and issue tracker sounds pretty logical. We can link
the new project from the RF main page similarly as e.g. RIDE.
Random note: Consider using 'testlink-' prefix for tags instead of
'testlink_'. We'll probably add enhancements to how tags are shown on
the report in RF 2.6 and tags having format 'prefix-value' will be
handled specially. We could, for example, automatically fold tags
having the same prefix in the statistics table, and create new tables
where the prefix is a column header.
Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org
cdekter
> Random note: Consider using 'testlink-' prefix for tags instead of
> 'testlink_'. We'll probably add enhancements to how tags are shown on
> the report in RF 2.6 and tags having format 'prefix-value' will be
> handled specially. We could, for example, automatically fold tags
> having the same prefix in the statistics table, and create new tables
> where the prefix is a column header.
the test case name to match up my Robot tests to SpiraTest test case
records (via their SOAP API). I had been considering moving the test
case numbers into tags, but this meant losing the easy referencing in
the report, since the tags aren't printed out as clearly as the test
case name. If I could have a column with say the 'id' tag for each
test case in the report, that would be fantastic.
Adam Sandman
Pekka Klärck
This looks like a nice tool for people using Spira and Robot
Framework. You may want to announce it also on other channels such as
our Slack or Forum, both linked from robotframework.org.
I noticed this tool processes output.xml files. As you may know, their
format is going to change in the forthcoming Robot Framework 7.0
release. For more information about the changes see the release notes:
https://github.com/robotframework/robotframework/blob/master/doc/releasenotes/rf-7.0rc2.rst#changes-to-output-xml
If the changes affect your tool, you can instruct users to use the
`--legacy-output` option until the tool is updated.
Cheers,
.peke
ke 3. tammik. 2024 klo 23.32 'Adam Sandman' via robotframework-users
(robotframe...@googlegroups.com) kirjoitti:
> To view this discussion on the web visit https://groups.google.com/d/msgid/robotframework-users/28ff376b-b717-4374-9865-852d19b7129an%40googlegroups.com.
Adam Sandman
|
CEO | Inflectra Corporation |