Rerunning Failed tests using DataProvider...

[Viral Sangani]

Hi There,

We're looking for a capability that when tests using Data Provider fail, the testng-failed.xml should generate a block for only the failed test with specific parameters/data sets & not all of the tests passing data through the same method. Currently testng-failled.xml reports entire test as failed. So we cannot run only the failed tests. Is there any ways to workaround this?

Here is a sceario.

 @Test(dataProvider="ProfilePostGetValidPattern")
public void testPattern1(String param1, String param2){
//use the parameters & do web service test request
}

 @DataProvider(name="ProfilePostGetValidPattern")
  public Object[][] createData() {

//generate object to be returned

return ret;
           }
}

In the above case, when one of the test requests fails, it logs the entire testPattern1 as failed. In case, if we have lots of tests using this same pattern, we won't be able to rerun the tests using testng-failed.xml. This is mostly blocking the use of DataProviders since rerunning only part of the data sets is not possible right now.

Is there a way to achieve this? Can we request this feature for the upcoming patch/release?

Thanks a lot,
Viral

[Cédric Beust ♔]

Anyone else interested in this feature?

Right now, the XML format doesn't let you specify a certain ordinal number for an invocation, but I would consider adding this if there's interest (right now, I think this would only be useful for testng-failed.xml but it might come in handy for testng.xml as well).

-- 

Cedric

--

You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--

Cédric

[sunith...@gmail.com]

Hi Cedric,

Yes it would be good to add this feature. It would be great if we can
execute only the failed datasets instead of rerunning the test for all
datasets in the dataprovider.

Thanks
Sunitha

> > testng-users...@googlegroups.com<testng-users%2Bunsu...@googlegroups.com>

> > .
> > For more options, visit this group at
> >http://groups.google.com/group/testng-users?hl=en.
>
> --

> ***Cédric
> *

[Cédric Beust ♔]

Hi Viral,

I just finished implementing this feature:  when certain invocations of a data provider fail, only these invocations will be run when you run testng-failed.xml

Can you try and let me know if it works for you?

You can download the current beta at:

http://testng.org/beta

Thanks!

--
Cedric

To unsubscribe from this group, send email to testng-users...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--

Cédric

[joevgear]

Hi Cedric,

Great feature as we may have test that fail due to network, slow UI
etc... and we only want to re-run the failed ones. I just tried the
beta and it is working, thanks for feature as we are using this in
our UI automation framework and re-running failed tests is KEY.

On Jan 24, 6:23 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> Hi Viral,
>
> I just finished implementing this feature:  when certain invocations of a
> data provider fail, only these invocations will be run when you run
> testng-failed.xml
>
> Can you try and let me know if it works for you?
>
> You can download the current beta at:
>
> http://testng.org/beta
>
> Thanks!
>
> --
> Cedric
>

> On Mon, Jan 11, 2010 at 3:12 PM, sunithayel...@gmail.com <

> > <testng-users%2Bunsu...@googlegroups.com<testng-users%252Buns...@googlegroups.com>

[Cédric Beust ♔]

Glad it's working for you, and thanks for testing it!

--
Cedric

To unsubscribe from this group, send email to testng-users...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--

Cédric