Script to get total number of test cases across test suites

[Chinmaya Karve]

Hi All,

Firstly - A BIG THANKS to all the folks who made Robot Framework and continue to make it better ! I have no words to express how it has eased our lives, and improved productivity of the entire team - thanks!

As our test suites are growing, I would like to know if there is a way to get the total number of test cases added across test suites? I know that this question has been asked before, but, I did not see a good answer - so posting it again. My objective is to run this "script" on a periodic basis and generate some numbers to be presented in management reports:

#	Test Suite	Test File	# of test cases
1	TS 1	TF1	10
2	TS 1	TF2	10
3	TS 2	TF3	10
		Total	30

Is something like this already available? Has someone already made such a script? if yes, please let me know.

Thanks!

Chinmaya

[Pekka Klärck]

Hello,

and thanks for kind words! Compliments are always appreciated. =)

If getting the number of tests after execution is enough, you can easily get that information from the output.xml files. You can parse the whole file easily with Robot's public result parsing modules, but if you just are interested in counts, you can get it using any XML parsing library from the `<statistics>` section at the end.

If you'd like to get the number of tests without executing them, you can try using the built-in TestDoc tool, run tests with the --dryrun opton, or use the public test data parsing modules.

Cheers,

    .peke

--
You received this message because you are subscribed to the Google Groups "robotframework-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to robotframework-users+unsub...@googlegroups.com.
To post to this group, send email to robotframework-users@googlegroups.com.
Visit this group at https://groups.google.com/group/robotframework-users.
For more options, visit https://groups.google.com/d/optout.

--

Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

[Chinmaya Karve]

Hi Pekka,

Yeah; my objective is to get the numbers without having to run the suite. I had the testdoc tool, but it gives keyword information etc also, whereas, I was looking only for numbers per test suite. Are there any public test data parsing modules that you could suggest for this?

Right now I am modifying the testdoc script to exclude keyword information from the output, and got initial success - can this be a switch in the tool in future?

Thanks again for suggestions!

Chinmaya

[zhuy...@gmail.com]

Hi

You can use --prerunmodifier or --listener to monitor your test execution.

And combine them with --dryrun which will avoid actual execution of tests.

e.g

pybot --listener my_listener.py --dryrun  my_suite

在 2017年11月23日星期四 UTC+8下午4:44:33，Chinmaya Karve写道：

[Pekka Klärck]

2017-11-23 10:44 GMT+02:00 Chinmaya Karve <cka...@gmail.com>:
>
> Yeah; my objective is to get the numbers without having to run the suite. I had the testdoc tool, but it gives keyword information etc also, whereas, I was looking only for numbers per test suite. Are there any public test data parsing modules that you could suggest for this?

You can either use robot.api.TestData factory method to just parse the
data or robot.api.TestSuiteBuilder to also construct an executable
test suite based on the data. The latter is probably more useful as
it's easier to inspect the model it's produces using. You could try
something like this:

from robot.api import TestSuiteBuilder, SuiteVisitor

class PrintTestCounts(SuiteVisitor):
def start_suite(self, suite):
print('{}: {}'.format(suite.name, len(suite.tests))

suite = TestSuiteBuilder().build('path/to/tests')
suite.visit(PrintTestCounts())

If you'd just be interested in the total number of tests, you could
just use `suite.test_count`. For more information about these APIs see
http://robot-framework.readthedocs.io/.

> Right now I am modifying the testdoc script to exclude keyword information from the output, and got initial success - can this be a switch in the tool in future?

Possibly. Depends on what it actually does.

Cheers,
.peke

[Chinmaya Karve]

Thanks Zhuy, trying your suggestion now...will let you know if it works!

On Thursday, 23 November 2017 11:08:46 UTC+5:30, Chinmaya Karve wrote:

[Chinmaya Karve]

Hi Pekka,

The way that we have output format options in libdoc, we can possibly have for testdoc,; so that one could create an xml file for easy parsing?

Another one could be similar to --flattenkeyword, where I do not want keyword information in testcase output, but just number of test cases found in each suite without further information.

Chinmaya

On Thursday, 23 November 2017 11:08:46 UTC+5:30, Chinmaya Karve wrote:

[Pekka Klärck]

2017-11-23 11:11 GMT+02:00 Chinmaya Karve <cka...@gmail.com>:
>
> The way that we have output format options in libdoc, we can possibly have for testdoc,; so that one could create an xml file for easy parsing?

Not sure is that worth it when you can parse tests directly using
Robot's public APIs or execute tests with --dryrun to get XML you can
parse.

[Krzysztof Jozefowicz]

Alternatively to Pekka suggestion, you may have a look into rflint, static analysis tool which have a method (which uses Robot internals) to get extract testcases,suites and keywords from robot file.
https://github.com/boakley/robotframework-lint

For examples, have a look into: https://github.com/nokia/RED/blob/master/misc/nilsimsa_POC/nilsimsa_POC.py ,particularly parse_rf_file method.

[Bryan Oakley]

Alternatively to Pekka suggestion, you may have a look into rflint, static analysis tool which have a method (which uses Robot internals) to get extract testcases,suites and keywords from robot file.

Actually, rflint doesn't use robot internals. It has its own parser (it's a little faster, plus it retains line number information and returns a much simpler data structure)

Here's a simple example of walking the tree of suites and test cases in order to print out all of the test case names and the line numbers where they are defined: https://github.com/boakley/robotframework-lint/wiki/Using-the-parser-to-walk-a-suite

On Thu, Nov 23, 2017 at 4:31 AM, Krzysztof Jozefowicz <kjo...@gmail.com> wrote:

Alternatively to Pekka suggestion, you may have a look into rflint, static analysis tool which have a method (which uses Robot internals) to get extract testcases,suites and keywords from robot file.
https://github.com/boakley/robotframework-lint

For examples, have a look into: https://github.com/nokia/RED/blob/master/misc/nilsimsa_POC/nilsimsa_POC.py ,particularly rfitem method.

--