New feature: @Test(skippingExceptions=...)

[Guillaume Juillot]

Hi,

I'm using TestNG for some years on functional tests and really like its power for this.
However there's few things I would really appreciate as an user, and @Test(skippingExceptions=...) is one of them.

I already forked TestNG and made the modifications myself. That's the first time I contribute to TestNG so I don't know the "normal" process.
That's why I post here before making the Pull Request.
You can check my changes in this branch https://github.com/gjuillot/testng/tree/skippingExceptions
I'm waiting for your comments before Pull Request.

Here is the description of my modifications.
I added a parameter skippingException to the @Test, similarly to the expectedExceptions.
If an exception in this list is thrown, the test will be marked as skipped.

For example during my tests, I need a remote connection to a device. If the connection fails, an exception is thrown and the test fails also.
The problem is that in this case, the failure is a false status because the connection outage prevents us from deciding if the system under test really does not pass the test.
With skippingExceptions, we can now mark such a test as skipped if a network failure occurs.

[Cédric Beust ♔]

Hi Guillaume,

You can already do this by throwing a SkipException. I'm not sure I see the point in allowing other exception classes to do this, can you explain why SkipException is not enough for you?

Thanks.

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-dev/-/bvpaH-wBPQkJ.
To post to this group, send email to testn...@googlegroups.com.
To unsubscribe from this group, send email to testng-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.

[Guillaume Juillot]

I know I can force a test to be Skipped by throwing an exception.
What I mean is that sometimes I make a call to an external library, that throws an Exception. In my case, because of that exception I can't conclude on the real result of the test.

For example:

    @Test
    public void skyShouldNotBeGreen() {
        WindowShutter.open(); // may throw a BlockedException
        assertNotEquals(Sky.color(), GREEN);
    }

I need to open the window shutter in order to see the color of the sky. If I can't open that shutter (open() throws a Exception), the test will fail. But that doesn't mean the sky is green.

-- 

Cédric

To unsubscribe from this group, send email to testng-dev+unsubscribe@googlegroups.com.

[Davide Del Vento]

How about this?

    @Test
    public void skyShouldNotBeGreen() {
        try {
            WindowShutter.open();
        } catch (BlockedException b) {
            throw SkipException("Can't tell, the shutter was blocked");
        }
        assertNotEquals(Sky.color(), GREEN);

[Cédric Beust ♔]

Yup, that's exactly what I was going to suggest.

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.

To post to this group, send email to testn...@googlegroups.com.

To unsubscribe from this group, send email to testng-dev+...@googlegroups.com.

[Guillaume Juillot]

That's what I'm doing for the moment but it's boilerplate code, isn't it ?

Le jeudi 5 juillet 2012 19:54:50 UTC+2, Cédric Beust ♔ a écrit :

Yup, that's exactly what I was going to suggest.

-- 

Cédric

On Thu, Jul 5, 2012 at 9:14 AM, Davide Del Vento <davide.d...@gmail.com> wrote:

How about this?

    @Test
    public void skyShouldNotBeGreen() {
        try {
            WindowShutter.open();
        } catch (BlockedException b) {
            throw SkipException("Can't tell, the shutter was blocked");
        }
        assertNotEquals(Sky.color(), GREEN);

    }

On Thu, Jul 5, 2012 at 9:49 AM, Guillaume Juillot <guillaum...@gmail.com> wrote:

    @Test
    public void skyShouldNotBeGreen() {
        WindowShutter.open(); // may throw a BlockedException
        assertNotEquals(Sky.color(), GREEN);
    }

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.

To post to this group, send email to testn...@googlegroups.com.

To unsubscribe from this group, send email to testng-dev+unsubscribe@googlegroups.com.

[kanam...@googlemail.com]

What about the option to skip Exceptions thrown by Testng itsself?

Something like:

Test(groups = { "manual" }, timeOut = 5000, skipExceptions = { org.testng.internal.thread.ThreadTimeoutException.class })

[Michael May]

I like this idea. It's better than having a try/catch block to catch exceptions just to rethrow a SkipException. Removing this boilerplate code would make the code more readable.

[Michael May]

I would use this for SQLTimeoutException's. I would rather tests that are skipped in this way show up as warnings instead of looking like any other skipped tests. I see these as being a different test type.

Test Types:
PASS - tests that pass
SKIP - skip because of a known issue, wrong environment - these tests weren't expected to run
WARN - skip because test timeout, etc; but these tests we're expected to run
FAIL - tests that fail

On Friday, February 1, 2013 8:54:39 AM UTC-5, kanam...@googlemail.com wrote: