@DataProvider on @BeforeMethod, or general state considerations

9,670 views
Skip to first unread message

Jeff Schnitzer

unread,
Feb 8, 2010, 9:16:40 PM2/8/10
to testng-users
Hi there! Despite being a longtime TestNG user, I need some advice.

I'm working on the unit tests for Objectify-Appengine, which is a "mid-
level" persistence API for Google App Engine/Java (lower level than
JDO, higher level than the Low-Level API). Our unit tests are built
with TestNG and all roughly work like this:

* Set up the environment in a @BeforeMethod method.
* Run some data manipulation commands.
* Verify the data.

Pretty run-of-the-mill stuff, and it works great. However, I just
added a caching layer to Objectify which users can enable on a class
by class basis. Now I want to run all my unit tests twice, once with
caching enabled and once with caching disabled.

I thought I might be able to use a @DataProvider with my "Set up the
environment" @BeforeMethod, but that doesn't seem to be available.

What is the recommended solution for this pattern? Surely this must
have come up before - there is some bit of state in the system that
could possibly produce different behavior, so all unit tests need to
be run at each state. Adding @DataProvider parameters to every single
test method would be onerous.

Thanks,
Jeff

Cédric Beust ♔

unread,
Feb 8, 2010, 10:12:41 PM2/8/10
to testng...@googlegroups.com
Hi Jeff,

You are correct that @Before methods cannot have a data provider.  The reason is that it would not make a lot of sense in terms of life cycle:  suppose that your data provider returns two sets of values, what does it mean that each @BeforeMethod will now be run two times?  And what does it mean for the test method itself?

The traditional way to approach this problem is to use @Factory.  In your factory, you now create one instance of your test class per set of values, the constructor of your class stores these values in fields and you can then use them in your @Before methods whenever you need them.  This preserves the semantics of all the configuration methods and it should solve your problem completely.

What do you think of this approach?

--
Cédric


 


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.




Jeff Schnitzer

unread,
Feb 9, 2010, 11:41:44 AM2/9/10
to testng...@googlegroups.com
2010/2/8 Cédric Beust ♔ <cbe...@google.com>:

> Hi Jeff,
>
> You are correct that @Before methods cannot have a data provider.  The
> reason is that it would not make a lot of sense in terms of life cycle:
> suppose that your data provider returns two sets of values, what does it
> mean that each @BeforeMethod will now be run two times?  And what does it
> mean for the test method itself?

I had hoped that the semantic for @DataProvider on @BeforeMethod would
mean that every test that uses the @BeforeMethod would run N times,
once with the @BeforeMethod called with the first parameter, once with
the @BeforeMethod called with the second parameter, etc... obviously
there's potential for a multiplicative explosion with multiple
@BeforeMethods and the @DataProvider on the test method itself, but
this is exactly what would be most useful, no? Essentially:

class MyTests {
@DataProvider(name="states")
public Object[][] generateStates() { ... }

@DataProvider(name="values")
public Object[][] generateValues() { ... }

@BeforeMethod(dataProvider="states")
public void setUp(String state) { ... }

@Test(dataProvider="values")
public void testSomething(String value) { ... }
}

Ideally this would end up testing every value for every state. In
practice, the @BeforeMethod would be on a base class and there might
be hundreds of tests with varying data providers that all would be
tested with each state.

> The traditional way to approach this problem is to use @Factory.  In your
> factory, you now create one instance of your test class per set of values,
> the constructor of your class stores these values in fields and you can then
> use them in your @Before methods whenever you need them.  This preserves the
> semantics of all the configuration methods and it should solve your problem
> completely.
>
> What do you think of this approach?

It is less satisfying than what I described above, but modifying every
class beats the hell out of modifying every single test method.

Can I put @Factory methods on the test classes themselves? If there's
a chicken-and-egg problem with the constructor, can I put the @Factory
method on a static inner class of the test class? Will these @Factory
methods get picked up by Eclipse automatically or do I need a
testng.xml file to specify the factories? The documentation seems
unclear on this point.

I'm trying to avoid a single class that must instantiate all the other
test classes, since it will inevitably get out of sync with multiple
people working in the project.

Thanks!

Jeff

Jeff Schnitzer

unread,
Feb 12, 2010, 4:01:04 PM2/12/10
to testng...@googlegroups.com
Hi folks... sorry to be a nag, but is there any chance someone could
comment on my questions about @Factory?

Thanks!
Jeff

Cédric Beust ♔

unread,
Feb 12, 2010, 4:39:01 PM2/12/10
to testng...@googlegroups.com
Hi Jeff,

Yes, you can put these @Factory classes anywhere, including on test classes.  TestNG will make sure not to loop forever trying to resolve them...

-- 
Cedric


--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.




--
Cédric


Iakiv Kramarenko

unread,
Aug 10, 2013, 1:25:55 AM8/10/13
to testng...@googlegroups.com, je...@infohazard.org
Hi, 

I was interested in the same thing, and this group conversation was the first I found. I got the idea but this topic lacks a good example of the subj. Just for others to find more quickly - here is a good example: http://www.automatedsqa.com/2012/05/testng-dataprovider-with-factory-annotation-sample-code.html

Best Regards,
Iakiv

Ankur Sharma

unread,
Apr 9, 2015, 9:16:34 PM4/9/15
to testng...@googlegroups.com
Hi Cedric,

The way we have "annotation..setDataProvider" for ".setDataProvider" , Do we have something where I could do the same  for "IConfigurationAnnotation" using "Factory" ?

In the listener, I have done this -

ITestAnnotation annotation

 annotation.setDataProvider("getTestData");
    annotation.setDataProviderClass(TestDataProvider.class);

I wish to do this :

IConfigurationAnnotation annotation

 annotation.setDataProvider("getTestDataBefore");
    annotation.setDataProviderClass(TestDataProvider.class);

or

 annotation.setFactory("getTestDataBefore");
    annotation.setFactoryClass(Factory.class);

Could you please help me, if you have any way to do this?

Rahul Rana

unread,
Aug 23, 2016, 12:07:57 PM8/23/16
to testng-users
Hi Cedric,

I have one @BeforeSuite method where in I am initializing something. I have one @Dataprovider method as well. What I want is to run @BeforeSuite first and them @dataprovider but it running in opposite way. ie. @dataprovider is running first and then @BeforeSuite.
Please suggest me a way to handle this situation.

Krishnan Mahadevan

unread,
Aug 23, 2016, 12:34:54 PM8/23/16
to testng-users

Can you please show us the code so that we can try and execute it to recreate the problem ?

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/




To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.

Rahul Rana

unread,
Aug 23, 2016, 1:56:25 PM8/23/16
to testng...@googlegroups.com
Hi Krishnan,

Below is the code snippet. To brief you about here in @BeforeSuite method I am cloning a repository from git-stash and in @DataProvider method I need to use that repo. But here @Dataprovider is running first and then @Beforesuite.

================================================Code======================================================
@BeforeSuite
public void cloneRepo(){
boolean IsRemoteURL = false;     
System.out.println("I am cloning the repo");
        if(System.getProperty(RunParameter.REMOTE_URL.getPropertyName()).equalsIgnoreCase("null")||System.getProperty(RunParameter.REMOTE_URL.getPropertyName()).equalsIgnoreCase("")){
              IsRemoteURL = true ; 
        }
       
        if(!IsRemoteURL){
  
String REMOTEURL = System.getProperty(RunParameter.REMOTE_URL.getPropertyName());
String BRANCHNAME  = System.getProperty(RunParameter.CODE_VERSION.getPropertyName()); 
String LocalPath = EnvironmentalVariables.DEFAULT_TEST_CASE_FILE_FOLDER
                                  + System.getProperty(RunParameter.CODE_VERSION.getPropertyName());
   
GitHub.getRepoClone(REMOTEURL,BRANCHNAME , LocalPath);
  }
}




@DataProvider(name = "getSuitesFromExcel", parallel = true)
public  Object[][] getSuitesFromExcel() throws InvalidFormatException, IOException {
System.out.println("I am dataprovider");
ArrayList<String> regionSuites = new ArrayList<>();
   ArrayList<String> executingTestSuites = new ArrayList<>();
String repoPath=EnvironmentalVariables.DEFAULT_TEST_CASE_FILE_FOLDER
+ codeVersion;
int count = 0;
if(executionType.equalsIgnoreCase("BVT")){
File dir = new File(repoPath + System.getProperty("file.separator") + executionType);
logger.info("Directory path: " +dir); 
File[] testScripts=dir.listFiles();
if(testScripts.length==0){
logger.error("Directory is empty");
}else{
for (int i=0;i<testScripts.length;i++){
executingTestSuites.add(testScripts[i].toString());
}
}
}else if(executionType.equalsIgnoreCase("Regression")){
// String S=RunParameter.SET_COMPONENT.getDefaultValue();
String components=System.getProperty("automation.setComponent");
logger.info("Running components are: "+components);
if(components==null || components==""){
logger.error("None of the component is selected. Please select the appropriate component");
}else{
String[] icomp=components.split(",");
for(int j=0; j<icomp.length; j++){
File dir = new File(repoPath + System.getProperty("file.separator") + executionType
+System.getProperty("file.separator")
+icomp[j]);
logger.info("Directory path: " + dir); 
File[] testScripts=dir.listFiles();
if(testScripts.length==0){
logger.error("Directory is empty");
}else{
for (int i=0;i<testScripts.length;i++){
regionSuites.add(testScripts[i].toString());
}
}
int MAX_SIZE=regionSuites.size();
if(size<MAX_SIZE){
for(int i=size; i<MAX_SIZE;i++ ){
if(count<4){
executingTestSuites.add(regionSuites.get(i));
count++;
size++;
}
}
}else{
logger.info("Completed executing all suites");
}
}
}
// TODO - add sendEmail in Utilities.jar
// TODO send email for regression - email info as logger.info ("" +
// executingTestSuites)
// TODO sendEmail(toList, sunject, body)
// TODO semdEmail(getFromControlFile, executingTestSuites)
logger.info("Executing Test Suites: " + executingTestSuites);
Object[][] returnObject = new Object[executingTestSuites.size()][1];
for (int i = 0; i < executingTestSuites.size(); i++) {
returnObject[i][0] = executingTestSuites.get(i);
}
return returnObject;
}



Thanks,
Rahul Rana



To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "testng-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/testng-users/3Kny3qTVdmg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to testng-users+unsubscribe@googlegroups.com.

Krishnan Mahadevan

unread,
Aug 24, 2016, 3:30:05 AM8/24/16
to testng-users

Rahul

I was hoping that you would share a sample that can be used to execute and recreate the problem.

I am not able to recreate your problem when I created a simple test.

Have you tried creating a simple test which just uses sysout statements and see if you are able to recreate the problem ? If not then am afraid the bug perhaps lies in your logic.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/




To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages