Request for comments: Asserts, the Next Generation

[Cédric Beust ♔]

Hi everyone,

I started thinking about ways to improve TestNG's assert system to make it more flexible. In particular, there are two scenarios that I'd like to support:

• Soft asserts.
• Logging asserts.

The current system is very inflexible due to the fact that asserts are static methods on the Assert class, which makes it pretty much impossible to extend. In order to address this, I'm considering adding a new assert class currently called FlexibleAssert. As opposed to Assert, this class doesn't have any static methods: it's supposed to be an instance field (or several) in your test class. By default, this class provides all the assert methods that you are used to, but it also offers various extension points which, hopefully, should allow anyone to extend it in any possible way.

For example, here is the simplest usage (very similar to what Assert does today):

@Test

public class A {

  private FlexibleAssert m_assert = new FlexibleAssert();

  public void test1() {

    m_assert.assertTrue(true, "test1()");

  }

}

Here are the extension points that FlexibleAssert offers, which subclasses can override:

• onBeforeAssert()
• executeAssert()
• onAssertSuccess()
• onAssertFailure()
• onAfterAssert()

All these methods receive an instance of IAssert, which captures the assert that's about to be run:

public interface IAssert {

  public String getMessage();

  public void doAssert();

}

For example, here is what a class that logs each assert would look like:

public class LoggingAssert extends FlexibleAssert {

  private List<String> m_messages = Lists.newArrayList();

  @Override

  public void onBeforeAssert(IAssert a) {

    m_messages.add("Test:" + a.getMessage());

  }

  public List<String> getMessages() {

    return m_messages;

  }

}

How to use this class in your test:

@Test

public class A {

  private LoggingAssert m_assert = new LoggingAssert();

  public void test1() {

    m_assert.assertTrue(true, "test1()");

  }

  public void test2() {

    m_assert.assertTrue(true, "test2()");

  }

  @AfterClass

  public void ac() {

    System.out.println("Tests run in this class:" + m_assert.getMessages());

  }

}

Now let's take a look at the implementation of a "soft assert" class. We want each assert to just run but not throw an exception if it fails. Instead, we just want to record it and once we've reached the end of the class (or the suite), assert the whole result. Here a simple implementation of such a class. The idea is simply to override the executeAssert() and if an exception occurs, catch it, store it and carry on:

public class SoftAssert extends FlexibleAssert {

  private Map<AssertionError, IAssert> m_errors = Maps.newHashMap();

  @Override

  public void executeAssert(IAssert a) {

    try {

      a.doAssert();

    } catch(AssertionError ex) {

      m_errors.put(ex, a);

    }

  }

  public void assertAll() {

    if (! m_errors.isEmpty()) {

      StringBuilder sb = new StringBuilder("The following asserts failed:\n");

      boolean first = true;

      for (Map.Entry<AssertionError, IAssert> ae : m_errors.entrySet()) {

        if (first) {

          first = false;

        } else {

          sb.append(", ");

        }

        sb.append(ae.getValue().getMessage());

      }

      throw new AssertionError(sb.toString());

    }

  }

}

How to use it:

@Test

public class A {

  private LoggingAssert m_assert = new LoggingAssert();

  public void multiple() {

    m_assert.assertTrue(true, "Success 1");

    m_assert.assertTrue(true, "Success 2");

    m_assert.assertTrue(false, "Failure 1");

    m_assert.assertTrue(true, "Success 3");

    m_assert.assertTrue(false, "Failure 2");

    m_assert.assertAll();

  }

The result:

FAILED: multiple

java.lang.AssertionError: The following asserts failed:

Failure 2, Failure 1

at org.testng.SoftAssert.assertAll(SoftAssert.java:32)

Please let me know what you think of such a design. In particular, if you've ever wanted to use a more flexible version of the Assert class, let me know if these extension points would be enough to implement what you need (e.g. taking a screen shot every time a Selenium assert fails, etc...).

Thanks.

-- 

Cédric

[Gogo]

Nice one! This was missing for such a long time. 

One thing that will still be needed is the possibility to have assertions that will fail immediately your test method (e.g. because you think that if this check fails the execution will be pointless). E.g. 

  public void multiple() {

    m_assert.assertTrue(true, "Success 1");

    m_assert.assertTrue(true, "Success 2");

    m_assert.assertTrue(false, "Failure 1");

    m_assert.assertTrue_elseExit(false, "Failure 2");

    m_assert.assertTrue(true, "Success 3");

    m_assert.assertTrue(false, "Failure 3");

    m_assert.assertAll();

  }

In this case, since you will never reach the assertAll method you will rely only on the log printed by each of the assert status at the moment of the execution. 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Cédric Beust ♔]

You can actually do this yourself in a couple of ways:

1) Declare two different assert fields, one FlexibleAssert (that throws) and one LoggingAssert, and use them both.

2) Extend LoggingAssert and add the assert method you want

-- 

Cédric

[Cédric Beust ♔]

FYI, initial support for this is now in master, for those of you who want to play with it.

-- 

Cédric

[Cédric Beust ♔]

FYI, I renamed and moved this class to org.testng.asserts.Assert.

-- 

Cédric

[Roman Hiden]

Really good, I am missing logging assert in my current project

[bevan]

That would be great Cedric, a much needed feature :)

My only worry:

I hope that I will be able to instantiate FlexibleAssert from another class and keep a static reference to it that I can reuse in a TestListener.

(I use behaviour classes for test steps, with methods that match "spoken-level" steps, test methods are just a sequence of those steps. The test step methods are where the assertions live)

i.e

@Test

public class A {

  public void test1() {

    Customer.create("bob");

  }

}

public class Customer {

  private SoftAssert softAssert = new SoftAssert();

  public static void create(String name) {

    AppCustomer c = new AppCustomer();

    try {

      c.setName(name);

    } catch (AppCustomerNameException e) {

      //Assert.fail("customer already exists");

      softAssert.fail("customer already exists");

    }

  }

}

I guess I need to play with it out off the Master branch.

Bevan

[bevan]

oops, I got a bit mixed up.

I meant 

public static SoftAssert softAssert = new SoftAssert();

and then run .assertAll() from within an @AfterMethod, or from OnSuccess/Failure in a TestListener.

[Cédric Beust ♔]

Hi Bevan,

I don't see any reason why this wouldn't work, you can definitely use one static assert object for all your tests. Experiment with master and let me know how it works for you.

Note: the class name is probably going to change to "Assertion", based on the feedback I have received so far.

-- 

Cédric

[Roman Hiden]

I have one suggestion:
IAssert interface has only 2 methods:getMessage(); doAssert();

Would be nice to have in there getActual() and getExpected() methods. The reason I am doing that is I want create a log which clearly state that expecte=actual or expected<>actual

I have added these method to IAssert and reflect the changes in FlexibleAssert. Currently lots of code repetition but I could not find a cleaner way on the fly.

Please see attached files.

P.S sorry can't attache so I am just pasting it here, Sorry

public abstract interface IAssert {
    public abstract String getMessage();

    public abstract void doAssert();

    public abstract Object getActual();

    public abstract Object getExpected();
}


import java.util.Collection;
import java.util.Map;
import java.util.Set;

/**
 * An assert class with various hooks allowing its behavior to be modified by
 * subclasses.
 */
public class Assert {
    protected void doAssert(IAssert assertCommand) {
        onBeforeAssert(assertCommand);
        try {
            executeAssert(assertCommand);
            onAssertSuccess(assertCommand);
        } catch (AssertionError ex) {
            onAssertFailure(assertCommand);
        }
        onAfterAssert(assertCommand);
    }

    /**
     * Run the assert command in parameter. Meant to be overridden by
     * subclasses.
     */
    protected void executeAssert(IAssert assertCommand) {
        assertCommand.doAssert();
    }

    /**
     * Invoked when an assert succeeds. Meant to be overridden by subclasses.
     */
    @SuppressWarnings("unused")
    protected void onAssertSuccess(IAssert assertCommand) {
    }

    /**
     * Invoked when an assert fails. Meant to be overridden by subclasses.
     */
    @SuppressWarnings("unused")
    protected void onAssertFailure(IAssert assertCommand) {
    }

    /**
     * Invoked before an assert is run. Meant to be overridden by subclasses.
     */
    @SuppressWarnings("unused")
    protected void onBeforeAssert(IAssert assertCommand) {
    }

    /**
     * Invoked after an assert is run. Meant to be overridden by subclasses.
     */
    @SuppressWarnings("unused")
    protected void onAfterAssert(IAssert assertCommand) {
    }

    abstract private static class SimpleAssert implements IAssert {
        private final String m_message;

        public SimpleAssert(String message) {
            m_message = message;
        }

        @Override
        public String getMessage() {
            return m_message;
        }

        public Object getActual() {
            return null;
        }

        public Object getExpected() {
            return null;
        }

        @Override
        abstract public void doAssert();
    }

    public void assertTrue(final boolean condition, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertTrue(condition, message);
            }
        });
    }

    public void assertFalse(final boolean condition, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertFalse(condition, message);
            }
        });
    }

    public void assertFalse(final boolean condition) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertFalse(condition);
            }
        });
    }

    public void fail(final String message, final Throwable realCause) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.fail(message, realCause);
            }
        });
    }

    public void fail(final String message) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.fail(message);
            }
        });
    }

    public void fail() {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.fail();
            }
        });
    }

    public void assertEquals(final Object actual, final Object expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Object actual, final Object expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final String actual, final String expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final String actual, final String expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final double actual, final double expected,
            final double delta, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert
                        .assertEquals(actual, expected, delta, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final double actual, final double expected,
            final double delta) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, delta);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final float actual, final float expected,
            final float delta, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert
                        .assertEquals(actual, expected, delta, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final float actual, final float expected,
            final float delta) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, delta);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final long actual, final long expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final long actual, final long expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final boolean actual, final boolean expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final boolean actual, final boolean expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final byte actual, final byte expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final byte actual, final byte expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final char actual, final char expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final char actual, final char expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final short actual, final short expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final short actual, final short expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final int actual, final int expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final int actual, final int expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertNotNull(final Object object) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotNull(object);
            }

        });
    }

    public void assertNotNull(final Object object, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotNull(object, message);
            }
        });
    }

    public void assertNull(final Object object) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNull(object);
            }
        });
    }

    public void assertNull(final Object object, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNull(object, message);
            }
        });
    }

    public void assertSame(final Object actual, final Object expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertSame(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertSame(final Object actual, final Object expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertSame(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertNotSame(final Object actual, final Object expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotSame(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertNotSame(final Object actual, final Object expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotSame(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Collection<?> actual,
            final Collection<?> expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Collection<?> actual,
            final Collection<?> expected, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Object[] actual, final Object[] expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEqualsNoOrder(final Object[] actual,
            final Object[] expected, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert
                        .assertEqualsNoOrder(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Object[] actual, final Object[] expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEqualsNoOrder(final Object[] actual,
            final Object[] expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEqualsNoOrder(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final byte[] actual, final byte[] expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final byte[] actual, final byte[] expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Set<?> actual, final Set<?> expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Set<?> actual, final Set<?> expected,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected, message);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertEquals(final Map<?, ?> actual, final Map<?, ?> expected) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertEquals(actual, expected);
            }

            public Object getActual() {
                return actual;
            }

            public Object getExpected() {
                return expected;
            }
        });
    }

    public void assertNotEquals(final Object actual1, final Object actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    public void assertNotEquals(final Object actual1, final Object actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }

        });
    }

    void assertNotEquals(final String actual1, final String actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final String actual1, final String actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final long actual1, final long actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final long actual1, final long actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final boolean actual1, final boolean actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final boolean actual1, final boolean actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final byte actual1, final byte actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final byte actual1, final byte actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final char actual1, final char actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final char actual1, final char actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final short actual1, final short actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final short actual1, final short actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final int actual1, final int actual2,
            final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    void assertNotEquals(final int actual1, final int actual2) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    public void assertNotEquals(final float actual1, final float actual2,
            final float delta, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, delta,
                        message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    public void assertNotEquals(final float actual1, final float actual2,
            final float delta) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, delta);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    public void assertNotEquals(final double actual1, final double actual2,
            final double delta, final String message) {
        doAssert(new SimpleAssert(message) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, delta,
                        message);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });
    }

    public void assertNotEquals(final double actual1, final double actual2,
            final double delta) {
        doAssert(new SimpleAssert(null) {
            @Override
            public void doAssert() {
                org.testng.Assert.assertNotEquals(actual1, actual2, delta);
            }

            public Object getActual() {
                return actual1;
            }

            public Object getExpected() {
                return actual2;
            }
        });

[Cédric Beust ♔]

Hi Roman,

I just made this change in master, thanks for the suggestion. I'm not 100% happy with it since "actual" and "expected" don't make sense for all assertions (e.g. "fail", "assertNotEquals", etc...) but I understand the value for typical "assertEquals" cases.

Please try the version in master and let m know how how it works for you.

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/YoEkMGkMo3IJ.

[Jason McSwain]

Hey Everyone,

Just wanted to add my opinion on this:

What you are asking for assumes that the conditions you are validating are simple objects (primitives)... that log nicely...

Message                       

           Expected    Actual                                                 Status
Check service response, not         null         [String representation of the object]         Pass     
Status check OK                         true         true                                                      Pass
Status check, not                         false       true                                                      Pass

What if you are doing something like (Assert.assertEquals("fail msg", complexObjectA, complexObjectB), assuming these had toString() implemented the logging could be a nightmare.  In our Framework that we use, we have a class like

• public class MyAssert extends AssertJUnit {}

In that class we add any functionality that we want personally.  Here you could add your own custom Asserts, logging, and whatnot... (based on your wanted it logged / tracked in your own personal format).  That is what we do, and i would hate to have the effort duplicated, and printed twice.

I'm not sure that this request is a good thing to add to base TestNG.  specifically b/c of the complex object scenario noted above.

Just my 2 cents,

-Jason-

On Thu, Aug 30, 2012 at 11:20 AM, Arcadie C. <arcad...@gmail.com> wrote:

Hi Cedric,

I've been missing logging asserts with expected and actual for a long time, as we are required to log both expected and actual messages for every (passed and failed) asserts.

The changes you made look good, and I have a request.

Would you consider adding expected and actual for assertFalse, assertTrue, assertNull, assertNotNull as well?

After, we could use it as this:

assertNotNull(object, "Check service response, not")
assertTrue(statusFlag, "Status check is OK")
assertFalse(statusFlag, "Status check, not")

and log it as:

Message                                  Expected    Actual                                                 Status
Check service response, not         null         [String representation of the object]         Pass     
Status check OK                         true         true                                                      Pass
Status check, not                         false       true                                                      Pass

Please let me know if this makes sense to you.

I've added the getActual and getExpected methods in assertFalse, assertTrue, assertNull, assertNotNull and uploaded them to pastebin here:
http://pastebin.com/BsJvFA19

Thanks,
Arcadie.

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/ZsygtUtJ0ZIJ.

[Arcadie C.]

Jason, thanks for your reply,

I understand that complex objects would make the log unreadable, but I'm only asking that those methods (assertNull, assertFalse, etc.) just expose those arguments, and leave it to the implementers to decide what will they do with it.

You could ignore them in your implementation, and someone could profit from that and just log not null instead of the string representation.

What I'm asking is just to make the actual object (and the expected / unexpected null or true/false) available for assertFalse / assertTrue / assertNull and assertNotNull, nothing more.

Please let me know if I've misunderstood your answer,

Thanks.

[Cédric Beust ♔]

Hi Arcadie,

I'm a bit confused:

Would you consider adding expected and actual for assertFalse, assertTrue, assertNull, assertNotNull as well?

After, we could use it as this:

assertNotNull(object, "Check service response, not")
assertTrue(statusFlag, "Status check is OK")
assertFalse(statusFlag, "Status check, not")

They are already implemented like this:

  public void assertTrue(final boolean condition, final String message) {

  public void assertFalse(final boolean condition, final String message) {

  public void assertNotNull(final Object object, final String message) {

See org.testng.asserts.Assertion.

Isn't this what you wanted?

-- 

Cédric

[Arcadie C.]

Cedric,

Let me provide an example. What I would like is for this (from org.testng.asserts.Assertion):

  public void assertTrue(final boolean condition, final String message) {

    doAssert(new SimpleAssert(message) {
      @Override
      public void doAssert() {
        org.testng.Assert.assertTrue(condition, message);
      }
    });
  }

to be changed to this:

  public void assertTrue(final boolean condition, final String message) {

    doAssert(new SimpleAssert(message) {
      @Override
      public void doAssert() {
        org.testng.Assert.assertTrue(condition, message);
      }

      @Override
      public Object getActual() {
        return condition;
      }

      @Override
      public Object getExpected() {
        return true;
      }
    });

  }


I would like to have access to the getActual() and getExpected() for assertTrue, assertFalse, assertNull, assertNotNull, just as you added for the other asserts, when extending Assertion and overriding onAssertSuccess(IAssert assertCommand), onAssertFailure(IAssert assertCommand), etc.

These are the only asserts that return null for expected and actual in the IAssertCommand.

I've modified the following methods from org.testng.asserts.Assertion:

public void assertNull(final Object object, final String message)

public void assertNull(final Object object)

public void assertNotNull(final Object object, final String message)

public void assertNotNull(final Object object)

public void assertFalse(final boolean condition)

public void assertFalse(final boolean condition, final String message)

public void assertTrue(final boolean condition, final String message)

And added this one, that was missing:

public void assertTrue(final boolean condition)

Please take a look at the modified org.testng.asserts.Assertion class (attached) or here: http://pastie.org/private/vjvnc0yzeggy19xwiwefa

Thanks, I hope this makes sense,
Arcade.

[Arcadie C.]

Cedric,

A pull request for the changes requested in the previous email has been submitted: https://github.com/cbeust/testng/pull/278
Could you please review them, and if appropriate, include them in the next build?

Thank you,

[Cédric Beust ♔]

It's in 6.8 in the Assertions class.

-- 

Cédric

On Thu, Sep 27, 2012 at 9:58 AM, Rocio Montes <roxio....@gmail.com> wrote:

is there a build already for me to use this features??

Thanks!

[Cédric Beust ♔]

Like I said, it's in the Assertions class.

-- 

Cédric

On Thu, Sep 27, 2012 at 11:05 AM, Rocio Montes <roxio....@gmail.com> wrote:

Thanks Cedric!

i downloaded the version 6.8

I read your post about setting up the properties and selecting "use project testng jar" because the eclipse  plugin has 6.7, so i did that

and I imported 

import org.testng.FlexibleAssert; and I get a "cannot be resolved"

also.. import org.testng.asserts.*;  and I still cant find the FlexibleAssert class

Please help me out, I need this feature.

Thanks!

[Cédric Beust ♔]

Sorry, the class is org.testng.asserts.Assertion.

-- 

Cédric

[Cédric Beust ♔]

I apologize, the correct class is org.testng.asserts.Assertion.

-- 

Cédric

-- 

Cédric

On Fri, Mar 1, 2013 at 6:20 AM, Patryk Kulecki <kulecki...@gmail.com> wrote:

I do not know if I am doing something wrong but I cant get it works. There is no Assertions class in 6.8 version.

[Kavita Sharda]

Hi Cedric,

Is there a way to have both SoftAssertion and Logging both success and fail messages.

@Test

public class A {

  private SoftAssert m_assert = new SoftAssert();

  public void multiple() {

    m_assert.assertTrue(true, "Success 1");

    m_assert.assertTrue(true, "Success 2");

    m_assert.assertTrue(false, "Failure 1");

    m_assert.assertTrue(true, "Success 3");

    m_assert.assertTrue(false, "Failure 2");

    m_assert.assertAll();

  }

The result something like this

Success1

Success 2

Failure 1

Success 3

Failure 2.

instead of just this

FAILED: multiple

java.lang.AssertionError: The following asserts failed:

Failure 2, Failure 1

at org.testng.SoftAssert.assertAll(SoftAssert.java:32)

[sunny sachdeva]

Yes you could do by extending org.testng.asserts.SoftAssert

AND override onAssertFailure and onAssertSuccess

@Override
 public void onAssertFailure(IAssert a, AssertionError ex)
 {
 try {
 test.log(LogStatus.FAIL, "<pre><b><font color='red'>" +"FUNCTIONAL FAILURE ::"+ex.getMessage()+"."+ "</font></b></pre>"+
 } catch (Exception e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 }
 }

@Override
 public void onAssertSuccess(IAssert a){
 try{
 test.log(LogStatus.PASS,"<b><font color='green'>"+a.getMessage()+"</font></b>");
 }catch(Exception e){


 }
 }

Hope this helps

~ Sunny