groups at the suite level

[Mehul Sanghvi]

Is it possible to exclude groups at the suite level ?

--
Mehul N. Sanghvi
email: mehul....@gmail.com

[Cédric Beust ♔]

On Wed, May 18, 2011 at 8:30 AM, Mehul Sanghvi <mehul....@gmail.com> wrote:

Is it possible to exclude groups at the suite level ?

I'm not sure I understand what you're asking, can you give an example?

-- 

Cédric

[Mehul Sanghvi]

2011/5/23 Cédric Beust ♔ <ced...@beust.com>:

Cedric,

Let me see if I can clarify it. Here is a sample of what we
have now in our testNG.xml:

<suite name="Checkins" verbose="3" >
<!-- Since test input parameters cannot be defined on per-class
basis, only at the Suite and the Test level,
we define Tests elements and thus avoid parameter name clashing.
-->
<test name="Core" preserve-order="true">
<groups>
<run>
<exclude name="non-standard"/>
<exclude name="performance"/>
<exclude name="functional"/>
</run>
</groups>
<classes>
<!-- does not take wild cards (would be nice to have that) -->
<class name="com.fubar.utils.TestTimestamp"/>
<class name="com.fubar.utils.TestUtils"/>
<class name="com.fubar.datamodel.TestSyncObject"/>
<class name="com.fubar.datamodel.TestArrayList"/>
<class name="com.fubar.datamodel.TestHashMapList"/>
<class name="com.fubar.datamodel.TestLinkedHashMapList"/>
<class name="com.fubar.datamodel.TestSortedArrayList"/>
</classes>
</test>

<test name="Personnel" preserve-order="true">
<groups>
<run>
<exclude name="non-standard"/>
<exclude name="performance"/>
<exclude name="functional"/>
</run>
</groups>
<classes>
<!-- does not take wild cards -->
<class name="com.fubar.accessmodel.TestAccess"/>
</classes>
</test>

</suite>

If you see above, the <groups> that are excluded are the same and are
repeated over and over again. It would be nice to be able to do that
sort of thing at the suite level:

<suite>

<groups>

<exclude name="non-standard" />
<exclude name="performance" />
<exclude name="functional" />

</groups>

<test ........ >

</test>

<test ..........>

</test>

</suite>

This makes it easier to have a "default" group that all tests belong
to. I may be getting my terminology wrong as I just
started with TestNG. More then that I'm acting as the gateway for
other people within the development group so there is
bound to be something lost in the translation, but hope this helps.

Essentially I'm trying to get a framework on top of TestNG that allows
the developers to write and run tests/groups of tests
much more easily and simply. In order to do that, any @Test notation
implies a default group called "check-in" otherwise
the test has an explicitly named group. I don't see a way to run a
suite of tests and say "for this suite I want you to use only
tests that belong to the default group, not any of the explicityl
named groups like "non-standard" "performance" "functional".

[Cédric Beust ♔]

Hi Mehul,

Not at all, what you're asking makes perfect sense and it's actually fairly consistent with TestNG's overall XML philosophy, where you can define tags in <suite> that you can then override in <test>. Here, I just need to pull this definition from <test> into <suite>...

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Cédric Beust ♔]

Hi Mehul,

I just implemented this feature in the beta:  http://testng.org/beta

You can now specify a groups tag under <suite>:

<suite name="SingleSuite" verbose="2" parallel="false" thread-count="4"

    data-provider-thread-count="3">

  <groups>

    <run>

      <include name="a" />

    </run>

  </groups>

  <parameter name="n" value="42" />

Note that just like the annotation @Test(groups), groups specified this way are accumulative. In other words, if you include group "a" in <suite> and "b" in <test>, both "a" and "b" will be run.

Obviously, this applies to the exclusion of groups as well.

Let me know how this works for you.

-- 

Cédric

2011/5/23 Cédric Beust ♔ <ced...@beust.com>

[Mehul Sanghvi]

2011/5/23 Cédric Beust ♔ <ced...@beust.com>:
> Hi Mehul,
> I just implemented this feature in the beta:  http://testng.org/beta
> You can now specify a groups tag under <suite>:
>
> <suite name="SingleSuite" verbose="2" parallel="false" thread-count="4"
>
>     data-provider-thread-count="3">
>
>   <groups>
>
>     <run>
>
>       <include name="a" />
>
>     </run>
>
>   </groups>
>
>   <parameter name="n" value="42" />
>
> Note that just like the annotation @Test(groups), groups specified this way
> are accumulative. In other words, if you include group "a" in <suite> and
> "b" in <test>, both "a" and "b" will be run.
> Obviously, this applies to the exclusion of groups as well.
> Let me know how this works for you.
> --
> Cédric
>

Cedric,

Thank you for the very quick response. I will talk to my
colleague and see about
using the beta. This would be pretty awesome. One other thing that
would be nice would
be the ability to do preserve-order=true at the <suite> level as well.
Is this possible in the
new beta ?

cheers,

mehul

[Nalin Makar]

Cedric,

how does this affect the testng-results.xml generated by org.testng.reporters.XmlReporter? For groups specified for inclusion directly under a suite, do you create a dummy <test>?

-nalin

[Cédric Beust ♔]

Nalin,

I haven't measured all the implications yet (another one I need to look into is YAML, which should hopefully be a no-op). Creating a dummy test sounds like a good compromise.

-- 

Cédric

[Cédric Beust ♔]

By the way, because of the way inheritance works, these groups will automatically appear under the <test> tag. For example, if you have:

<suite>

  groups=a

  <test>

    groups=b

Calling getGroups() on the test will return both a and b, which means that testng-results.xml will display these groups, although it won't be clear which ones came from the suite... So in effect, it already "kind of works".

[Cédric Beust ♔]

2011/5/23 Mehul Sanghvi <mehul....@gmail.com>

Cedric,


      Thank you for the very quick response.  I will talk to my
colleague and see about using the beta.  This would be pretty awesome.  One other thing that
would be nice would be the ability to do preserve-order=true at the <suite> level as well.
 Is this possible in the new beta ?

<test> tags are already run in the order they are encountered in the XML file, so I don't believe such a flag is necessary. For more details, take a look at the comment I wrote on this JIRA issue. 

-- 

Cédric

[Nalin Makar]

ok. sounds good.

[Harish Appannagari]

Cedric,
This an old post and I am currently in need of this feature. The beta link provided below is not available any more. Is this feature of having to use groups at suite level available at all now ?

Regards,
-Harish

[Krishnan Mahadevan]

Harish,

Have you tried using it in the latest TestNG version ? If yes, what do you see ?

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.

Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Harish Appannagari]

I tried with :  compile 'org.testng:testng:6.9.10'
Note : I am not seeing an error but I see that tests are not getting executed as per my assumption.

Here is my sample xml suite :
<suite name="SingleSuite" >

    <groups>
        <run>
            <include name="group2" />
        </run>
    </groups>


    <suite-files>
        <suite-file path="Group1.xml" />
        <suite-file path="Group2.xml" />
        <suite-file path="Env.xml" />
    </suite-files>

</suite>

My Expectation : Only tests grouped as "group2" should execute.
But i see that all the tests in the suite files are getting executed.

Thanks,

-Harish

--
You received this message because you are subscribed to a topic in the Google Groups "testng-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/testng-users/a504gCYpT28/unsubscribe.
To unsubscribe from this group and all its topics, send an email to testng-users...@googlegroups.com.

[Harish Appannagari]

Any update for me on this issue ? I see that groups defined at the parent suite level do not flow to child suites.

Regards,

-Harish

[Krishnan Mahadevan]

Harish,

I hope you went through the thread completely.

This thread talks about supporting inclusion and exclusion of groups at <suite> level.

You are asking about something else.

What is currently available as implementation in the latest version of TestNG is the below ability :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="GroupSuite" parallel="false" verbose="3">

    <groups>
        <run>
            <include name="a"/>
        </run>
    </groups>

    <test name="GroupBTest" verbose="2">
        <groups>
            <run>
                <include name="b"/>
            </run>
        </groups>
        <packages>
            <package name="organized.chaos.testng.foo"/>
        </packages>
    </test>
</suite>

The above suite file will cause methods that below to groups "a" and "b" in all the test classes that reside inside organized.chaos.testng.foo package to be executed.

In the above example you can also define exclusions at the suite level and that would get applied for all the <test> tags in your suite file.

What you are asking for is NOT yet supported in TestNG to the best of my knowledge.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Moustakas42]

Is it in any way possible to define group dependencies at they suite level, bypassing test class hierarchy?