Launch same test class more than once, different parameters.

[H.]

Hi everybody,

I'm having some issues with launching the same test class more than once, with different parameters.
[b]The TestNG documentation shows that it should be possible[/b] to run the same class several times, from the same <test></test> in a suite. (here : http://testng.org/doc/documentation-main.html#factories )
So that's what I am doing because for some reasons the factory is not convenient for me (in a functional user's point of view. not technically).

So when running twice the same class but with different input parameters, it only executes one of them (surprisingly, the second one!).

Any ideas?
I'm using testng-5.5-jdk15.jar version.
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=160376#160376

[Cédric Beust ♔]

Again:  please post your code, we can't really help you otherwise...

--
Cedric

--
Cédric

[H.]

Hi cedric. Here my testng.xml code that is causing me harm....

<test verbose="2" name="Front-OrderProducts" annotations="JDK">

<classes>

<parameter name="customerFile" value="C:\\Run\\required\\input\\Customer2.properties"/>
<parameter name="fromLine" value="2"/>
<parameter name="toLine" value="4"/>
<class name="front.Do.FrontDoOrderProducts" />

<parameter name="customerFile" value="C:\\Run\\required\\input\\Customer2.properties"/>
<parameter name="fromLine" value="3"/>
<parameter name="toLine" value="3"/>
<class name="front.Do.FrontDoOrderProducts" />

</classes>
</test>

TestNG will execute only one of the two occurences, and always the second one, surprisingly and without apparent reason.

I may tell you also that I defined a @BeforeSuite test method that is therefore common to all my tests. The function is a function to instanciate my DefaultSelenium object upon which I run my test through.
Anyway I'm nearly sure this is not the problem, as I can launch several <test> in the same suite and they don't mix up or mess around even if using a same DefaultSelenium configuration (note that I'm using the *chrome mode that allow domains switching...).

Thanks for you help.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=160460#160460

[Moiz]

I think you should use
@BeforeClass: The annotated method will be run before all the tests in
the test class have been run.

Not @BeforeSuite

-Moiz

> Posted via Jive Forumshttp://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=1...

[H.]

I want the instructions to be common and shared to all the tests I defined in my <suite>. That's why it makes sense for me to use @BeforeSuite. Otherwise, you'd probably be right!

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=160475#160475

[Cédric Beust ♔]

Parameter names must be unique in a <test> stanza.  Same for classes.  Use two different <test> tags and it should work as expected.  Also, note that I extracted the customerFile parameter so that it can be shared by both <test> tags:

<parameter name="customerFile" value="C:\\Run\\required\\input\\Customer2.properties"/>

<test verbose="2" name="Front-OrderProducts" annotations="JDK">
  <classes>

    <parameter name="fromLine" value="2"/>
    <parameter name="toLine" value="4"/>
    <class name="front.Do.FrontDoOrderProducts" />

  </classes>
</test>

<test verbose="2" name="Front-OrderProducts" annotations="JDK">
  <classes>

    <parameter name="fromLine" value="3"/>
    <parameter name="toLine" value="3"/>
    <class name="front.Do.FrontDoOrderProducts " />
  </classes>
</test>

--
Cédric

[Alexandru Popescu ☀]

And last, but not least, for this kind of scenarios I think a better
fit is @DataProvider.

./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator

[Moiz]

You are right Alex. But the problem with @DataProvider is the if
someone who doesnt know Java can't change the parameters
and it need to compiled again.
So changing parameters in xml is more user friendly.

Thanks,
Moiz

On Jun 21, 11:41 am, "Alexandru Popescu ☀"

> > Cédric- Hide quoted text -
>
> - Show quoted text -

[Alexandru Popescu ☀]

On 6/21/07, Moiz <moiz...@gmail.com> wrote:
>
>
> You are right Alex. But the problem with @DataProvider is the if
> someone who doesnt know Java can't change the parameters
> and it need to compiled again.
> So changing parameters in xml is more user friendly.
>

This if you cannot load those external parameters in your
@DataProvider and feed them this way to your tests... but I guess you
got the picture already.

[H.]

Thank you Cedric,

I'll do that way, creating <test> each time I need to invoke my class. But you should modify the TestNG documentation and remove the example which apparently does not work. I saw this at the beginning of the section related to Factories.

And yes, to fuel the debate on Factory or not Factory, my choice was to rely on simple parameter inputs from the .XML document whilst not overloading my Java code with specific constructors and Factory classes...
It might be convenient for some specific situations, but in my case, taking into account that functional users will have to update the tests, it just makes sense to keep it all clear from a same testng.xml "source-file".

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=160780#160780

[Cédric Beust ♔]

On 6/22/07, H. <testng...@opensymphony.com> wrote:

Thank you Cedric,

I'll do that way, creating <test> each time I need to invoke my class. But you should modify the TestNG documentation and remove the example which apparently does not work. I saw this at the beginning of the section related to Factories.

Oh, good catch!  I never noticed these examples were incorrect.

I fixed it, please take a look at the updated version.

Thanks for finding this!

And yes, to fuel the debate on Factory or not Factory, my choice was to rely on simple parameter inputs from the .XML document whilst not overloading my Java code with specific constructors and Factory classes...
It might be convenient for some specific situations, but in my case, taking into account that functional users will have to update the tests, it just makes sense to keep it all clear from a same testng.xml "source-file".

Sure, that's fine, but keep in mind that if you switch to @Factory/@DataProvider at some point, you can then make these parameters even easier to update by putting them in, say, an Excel spreadsheet.

--
Cédric

[tom lichtenberg]

My tests all use the DataProvider
@Test(dataProvider = "config", timeOut = 10000)
and for the most part this has been fine. The "config" specified here is a Java object composed of a conglomeration of XML configuration files, which are loaded by my configuration manager into a Config object. Once loaded all of those values are static.

I want to be able to call the test method repeatdely, and tell it to use a different subConfig found within the overall config each time, and thought I would be able to do this using the Parameters annotation, but I found the two are mutually exclusive - I can use either DataProvider or Parameter but not both.

My idea was intended to work like this: (TestComponent.java)
@Parameters({ "param" })
@Test(dataProvider = "config", timeOut = 10000)
public void testLogin(Config config, String param) {
// here, load the subConfig of the Config based on the value of 'param'
... }

Where I could use the testng.xml as described above in this thread:
<suite name="component-tests">
<test verbose="2" name="Componentr" annotations="1.5">
<parameter name="param" value="def"/>
<classes>
<class name="TestComponent"/>
</classes>
</test>

<test verbose="2" name="Component" annotations="1.5">
<parameter name="param" value="def"/>
<classes>
<class name="TestComponent"/>
</classes>
</test>
</suite>

the test method would ideally be invoked with param=abc and param=def, however, the dataProvider annotation precludes the parameter annotation, and I get the error that the method is called with the wrong number of arguments (one instead of two)

I'd be glad of any advice on how to deal with this problem
regards,
tom

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=169154#169154

[Cédric Beust ♔]

I would suggest using an Annotation Transformer that would modify the value of the dataProvider parameter before the test method gets invoked.

Documentation: http://testng.org/doc/documentation-main.html#annotationtransformers

Please let us know if you need more help using this feature...

--
Cédric

[Alexandru Popescu ☀]

I am not sure I've understood the usage of the param. Is it meant to
determine a specific @DataProvider invocation or is it just another
value you want to pass to the @Test?

For the first scenario, you can probably pass to the @Test method the
@Parameter and then invoke manually a method that plays the role of
the @DataProvider. For the later, I think you can try to parameterize
the @DataProvider method to receive the @Parameter.

bests,

[tom lichtenberg]

Yes, "param" is "just another value I want to pass to the @Test"

You suggested, "I think you can try to parameterize

the @DataProvider method to receive the @Parameter."

but I am not clear what this means.

When I try to use the @Parameters notation on the dataProvider method, as below, I get java.lang.IllegalArgumentException: wrong number of arguments - when invoking the data provider method

@Parameters({ "param" })
@DataProvider(name="config")
public Object[][] getConfig(Method method, String param) throws ConfigurationException
{
return new Object[][] { new Object[] { getConfig(method.getName().toLowerCase()) }, param };
}

[testng] Exception in thread "main" java.lang.IllegalArgumentException: wrong number of arguments
[testng] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[testng] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[testng] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[testng] at java.lang.reflect.Method.invoke(Method.java:585)
[testng] at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:552)
[testng] at org.testng.internal.MethodHelper.invokeDataProvider(MethodHelper.java:592)
[testng] at org.testng.internal.Invoker.handleParameters(Invoker.java:636)
[testng] at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:715)
[testng] at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:105)
[testng] at org.testng.TestRunner.privateRun(TestRunner.java:682)
[testng] at org.testng.TestRunner.run(TestRunner.java:566)

testng.xml contains the parameter like this:
<suite name="dispatcher-tests">

<test verbose="2" name="Component" annotations="1.5">
<parameter name="param" value="def"/>
<classes>
<class name="TestComponent">

<methods>
<include name="testComponent" />
</methods>
</class>
</classes>
</test>
</suite>

and the test method like this:

@Test(dataProvider = "config", timeOut = 10000)

public void testComponent(Config config, String param) throws Exception
{

}
---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=169449#169449

[Cédric Beust ♔]

On 7/17/07, tom lichtenberg <testng...@opensymphony.com> wrote:

Yes, "param" is "just another value I want to pass to the @Test"

You suggested, "I think you can try to parameterize
the @DataProvider method to receive the @Parameter."

but I am not clear what this means.

When I try to use the @Parameters notation on the dataProvider method, as below, I get  java.lang.IllegalArgumentException: wrong number of arguments - when invoking the data provider method


   @Parameters({ "param" })
    @DataProvider(name="config")

You can't have both these annotations on a test method (we should give you a clearer error message).  I'm guessing that TestNG is picking only one (@Parameters) and since it only defines one parameter and your method expects two, you are receiving this error.

I'm not sure what you are trying to achieve, can you give us an example of a few test runs with different values?  Then we should be able to help you.

--
Cédric

[tom lichtenberg]

Our dataProvider is uesd to load a complex series of xml configuration files, conglomerating them into on Config object. Currently, to use different sets of variables within this object, we are using separate test methods. We would like to be able to use an independent parameter (outside of the data provider) to tell one test method which set of variables to use, effectively multitasking that test method.

testng.xml would only call the method "test", instead of "testOne", "testTwo", etc ..., given this data:

desired XML config:

<test>
<abc>
<name = "abc_name"/>
</abc>
<def>
<name = "def_name"/>
</def>
</test>

current XML configuration:

<testOne>
<name = "abc_name"/>
</testOne>

<testTwo>
<name = "def_name"/>
</testTwo>

in the test method, then, we would select which element within <test> to use for "name" (parameter is :abc" or "def", etc ...)

I hope that makes the issue a little clearer.

It's not a life or death issue. We can always use separate test methods - as stub-like entry points - serving only to load their parameters from the Config, and call the actual test with those. It just seems awkward

Thanks!

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=169506#169506

[Cédric Beust ♔]

How about implementing your data provider so that it would parse the XML config file you give below and then return these values?

For example (approximate syntax):

   @DataProvider(name="config")
   public Object[][] getConfig(Method method) throws ConfigurationException
   {
      String[] params = parseXmlConfig();
      List result = new ArrayList();
      for (String param : params) {
        result.add(new Object[] { getConfig(method.getName().toLowerCase(), param); });
     }
     return result.toArray();
   }

--
Cedric

On 7/17/07, tom lichtenberg <testng...@opensymphony.com> wrote:

--
Cédric

[jmartin]

Hello,

I am working with Tom on this effort. Let me try to explain what we are trying to achieve. We would like to figure out if there is a way that we can have one test method that we can invoke multiple times with different input values each time. We would like to be able to specify the multiple invocations of the single method in the testng.xml file, and we would like to be able to specify the different sets of input values (and expected results) for each of the invocations in perhaps another xml file. So far we have not been able to figure out how to do that with test NG. It seems like it would be a useful feature in that it would allow a developer to write one test method and then a black box tester could write several test cases providing different input values (and expected results) for each invocation by manipulating xml files only.

The limitation seems to be you can only associate one data provider with a test method, but there is no way (that we can find) to tell that data provider what set of input values for read and feed to the test method for each given invocation of the test method. If there was a way to pass the data provider the 'name' of the test that is being run, that would be enough additional information to allow it to pass the appropriate set of input values to the test method, but we don't think such a feature exists.

I hope this better explains what we are trying to achieve and any ideas you have to help us get there would be appreciated.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=169561#169561

[Alexandru Popescu ☀]

Got your explanations. Seems like an interesting/advanced test. Gonna
think about a solution. I'll keep you posted.

[jmartin]

Thanks for your reply. Other than repeating what I wrote though, I don't see any additional content in it. Is there something I am missing here?

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=92692&messageID=169969#169969

[Alexandru Popescu ☀]

On 7/18/07, jmartin <testng...@opensymphony.com> wrote:
>

> Thanks for your reply. Other than repeating what I wrote though, I don't see any additional content in it. Is there something I am missing here?

Not sure what are you refering to.

Here is a quick idea (not sure it will work or not):

@BeforeClass
@Parameters({"xmlPart"})
public void setXmlPart(String xmlPart) {
this.xmlPart = xmlPart;
}

@DataProvider()
public loadXmlPart() {
// use the this.xmlPart to decide
}

and the rest of the code should be the one you are already having.

bests,

./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator