Test Suite Naming Conventions
Martin Taylor
structure of test suites it names each successive level by prepending
the first character of the suite name and a period. For example, a
folder and file structure like this:
-- Foo_Suite
---- Bar_Suite
------ Test_Suite.html
-------- Foo Bar Test
would result in report entries looking like
Foo Suite
f.Bar Suite
f.b.Test Suite
f.b.t.Foo Bar Test
The only way I can see this being a useful feature is if you
established a suite naming convention where each suite name would be
prefixed by the next sequential letter of the alphabet. For example,
renaming the suites in the above example to:
-- A_Foo_Suite
---- B_Bar_Suite
------ C_Test_Suite.html
-------- Foo Bar Test
would result in report entries looking like
A Foo Suite
a.B Bar Suite
a.b.C Test Suite
a.b.c.Foo Bar Test
Is this what people generally do in practice, or is there some other
naming convention that is preferred?
Thanks,
Martin
Pekka Klärck
>
> When Robot Framework produces a report of a run of a nested folder
> structure of test suites it names each successive level by prepending
> the first character of the suite name and a period. For example, a
> folder and file structure like this:
>
> -- Foo_Suite
> ---- Bar_Suite
> ------ Test_Suite.html
> -------- Foo Bar Test
>
> would result in report entries looking like
>
> Foo Suite
> f.Bar Suite
> f.b.Test Suite
> f.b.t.Foo Bar Test
I assume you mean the Statistics by Suite table in reports and logs?
The reason behind this is that we want to somehow show how the suite
is structured. For example having just
Foo Suite
Bar Suite
Test Suite
Foo Bar Test
would not give too much information about the structure. I'm not that
happy with the current solution, though, and would like to hear if
anyone has ideas on how to make it better. Some kind of folding
structure like with suites/tests/keywords in log files would probably
be best but it requires some non-trivial JavaScript and getting layout
nice with all browsers isn't that easy. Perhaps having simple ASCII
art like below would be more clear than the current system:
Foo Suite
`-- Bar Suite
`-- Test Suite
`-- Foo Bar Test
> The only way I can see this being a useful feature is if you
> established a suite naming convention where each suite name would be
> prefixed by the next sequential letter of the alphabet. For example,
> renaming the suites in the above example to:
>
> -- A_Foo_Suite
> ---- B_Bar_Suite
> ------ C_Test_Suite.html
> -------- Foo Bar Test
>
> would result in report entries looking like
>
> A Foo Suite
> a.B Bar Suite
> a.b.C Test Suite
> a.b.c.Foo Bar Test
>
> Is this what people generally do in practice, or is there some other
> naming convention that is preferred?
I don't actually understand what is the benefit of such a naming
convention. Could you please clarify?
Cheers,
.peke
Ulrich
Is there any tool to create a test specification document from a
robotframework directory *before* test execution, i.e. a document
containing the suite and test names including all 'documentation'
elements?
If a separator existed for more flexibility each test suite/test case
could be prefixed by a 'section id'
Assuming a variation of Martin’s example structure and a double
underline character as separator:
Regressiontest Product Xyz
-- 01__Foo_Suite
---- 01__Bar_Suite
------ 01__Test_Suite.html
-------- 01__Test1
-------- 02__Test2
Leading zeroes were added as that ensures the directories/files are
ordered correctly when listed. In the generated test specification the
leading 0 could be automatically removed.
I’d like to see something like this as content table of the generated
test specification:
Regressiontest Product Xyz
1 Foo Suite
1.1 Bar Suite
1.1.1 Test Suite
1.1.1.1 Test1
1.1.1.2 Test2
The prefix could be a, b, c, d, … or i, ii, iii, iv, … or anything
else and the separator could be made configurable.
Thanks,
Ulrich
On Feb 15, 11:30 pm, Pekka Klärck <p...@iki.fi> wrote:
> 2009/2/11 Martin Taylor <cmtay...@ti.com>:
Pekka Klärck
>
> Looking at this issue more from a documentation standpoint:
> Is there any tool to create a test specification document from a
> robotframework directory *before* test execution, i.e. a document
> containing the suite and test names including all 'documentation'
> elements?
testdoc.py does exactly that:
http://code.google.com/p/robotframework/wiki/TestDataDocumentationTool
> If a separator existed for more flexibility each test suite/test case
> could be prefixed by a 'section id'
> Assuming a variation of Martin's example structure and a double
> underline character as separator:
>
> Regressiontest Product Xyz
> -- 01__Foo_Suite
> ---- 01__Bar_Suite
> ------ 01__Test_Suite.html
> -------- 01__Test1
> -------- 02__Test2
>
> Leading zeroes were added as that ensures the directories/files are
> ordered correctly when listed. In the generated test specification the
> leading 0 could be automatically removed.
This is an interesting idea. I would actually remove the whole prefix
after using it for sorting. You can obviously already now prefix
files/dirs how ever you want to get them sorted, but automatically
removing the prefix from the suite name would allow keeping nice
looking names. This would nicely solve the need to specify the
execution order of test suites too:
http://code.google.com/p/robotframework/issues/detail?id=89
> I'd like to see something like this as content table of the generated
> test specification:
>
> Regressiontest Product Xyz
> 1 Foo Suite
> 1.1 Bar Suite
> 1.1.1 Test Suite
> 1.1.1.1 Test1
> 1.1.1.2 Test2
> The prefix could be a, b, c, d, … or i, ii, iii, iv, … or anything
> else and the separator could be made configurable.
The doc generated by testdoc.py has similar nested structure as normal
log files and special prefixes aren't thus needed. A different tool
could, of course, use prefixes, and having them in Statistics By Suite
table would be one possibility too.
Cheers,
.peke