NUNit 3 ActionAttributes and SetUp / TearDown

[Paul Hicks]

I've been working around the current behaviour of ActionAttributes' BeforeTest vs. SetUp, and AfterTest vs. TearDown.
I've read the thread that has all the most useful commentary and I know that decisions for v3 aren't final yet.
I'm pretty sure this is an intractable problem, since I want it to work both various ways, depending on which test I'm looking at.

Is this a suitable forum to make a suggestion? Because I'd quite like ITestAction to get a couple of new methods, BeforeSetUp(ITest)
and AfterTearDown(ITest). I could get all my test cases working with the existing two methods and those two new ones :)

[Charlie Poole]

Hi Paul,

Right here is a good place to get a discussion going. It would be
helpful to have some use cases that demonstrate **why** those methods
might be useful. Otherwise, it ends to leave those of us not seeing
the point.

The current implementation isn't what I had hoped to have ready for
the release, but it's a starting point. As described on the wiki, we
wanted action attributes to run after setup and before teardown **for
each level of inheritance**. That's not how it's currently working. If
we get it working that way, your desired BeforeSetUp behavior could be
provided simply by placing the attribute on a base class. In many
ways, that seems preferable to doubling the number of interface
methods.

Note that you can also roll your own attribute, without the shortcut
of using an ActionAttribute. Documenting how to do that is probably a
higher priority for users than any change to ActionAttributes.
Unfortunately, we are all developers and tend to save the docs for
last. :-( Still, you might take a look at how some of our own
attributes are implemented. The same interfaces used are available to
you. The only thing that ActionAttributes add is the ability to
specify test-case behavior at the fixture or higher level. If
specifying behavior on the method itself is sufficient, you may simply
want to implement a custom attribute.

Charlie

> --
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nunit-discus...@googlegroups.com.
> To post to this group, send email to nunit-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/nunit-discuss.
> For more options, visit https://groups.google.com/d/optout.

[Paul Hicks]

Hi Charlie,

I'm hoping to get AOP-style cross-cutting behaviour out of ActionAttributes, which is why I haven't explored test class inheritance yet.
I'll explain my specific case.

Our database is updated by several separate applications: the web application, and various batch processes that accrue information to build up aggregated data used for reporting, journalling, building general ledger entries, and so forth.
For our tests, we want to clear out the batch processing tables that store process (rather than entity) state, effectively reseting our batch machines in a way that the batch processes cannot do (since it's not ever required outside of testing).
Currently, we have methods for setting up and tearing down around these tests, and we have to compose the SetUp and TearDown methods for each test suite depending on which processes it interacts with.
We'd like to be able to annotate tests with ActionActtributes like "GeneralLedgerCleanSlate" and "AnnualReportingCleanSlate" so that the SetUp method can rely on having a clean slate, no matter what previous tests blew up, leaving process tables in all sorts of unsavoury states. These attributes would have to be composable, which is why we don't want to use inheritance.

Using my own Attribute is definitely something I'd consider. I don't personally have an issue with two extra methods on the interface, especially since the TestActionAttribute class provides virtual implementations for me. Even separate interfaces, each with two methods, would work for me.

[Michael]

For each level of inheritance? Why not leverage language level features like polymorphism, virtual, override, etc? I've said that all along when it came to designing test fixture hierarchies. I am somewhat surprised by the intended functionality to be perfectly frank. Xunit for instance tends to be more language level by contrast. I happen to think that some level of attribution is desirable, but there comes a point that you're forcing the language to operate in ways it was never intended to do so. That's my two cents on the topic. HTH

[Charlie Poole]

Hi Michael,

I understand your comment in general, but not specifically as it
applies to this issue.

Are you saying the ActionAttribute feature should not exist? Or do you
have some idea for resolving this issue in a way that is more
consistent with the language? If it's the latter, I'd like to hear it
because, frankly, I'm not really happy about how ActionAttributes
work right now.

In general, NUnit has always tried to avoid language-based features,
which cannot be duplicated across all .NET languages. However, there
is also such a thing as .NET-supported polymorphism and we could
certainly leverage that a great deal more than we do. In xUnit, they
manage to stay more in line with C# polymorphism by not supporting
certain things that are not viewed as "good to do" for developers. In
practice, some such features end up as extensions. In any case, the
xUnit framework is much more strongly opinionated than NUnit is today
regarding how people should write tests. It's unlikely that either of
us will change in our overall outlook.

ActionAttributes, to the extent that they enable cross-cutting
aplication of code through attributes, is definitely outside the
normal OO paradigm. But that's also true of the SetUp and TearDown
attributes, which have been with us since NUnit 2.0 and are generally
accepted practice in many other frameworks. Once you have such
features, you have to figure out how they interact with normal
polymorphism and that's what the present issue is all about.

Charlie

On Wed, Dec 16, 2015 at 12:24 AM, Michael <mwpow...@gmail.com> wrote:
> For each level of inheritance? Why not leverage language level features like polymorphism, virtual, override, etc? I've said that all along when it came to designing test fixture hierarchies. I am somewhat surprised by the intended functionality to be perfectly frank. Xunit for instance tends to be more language level by contrast. I happen to think that some level of attribution is desirable, but there comes a point that you're forcing the language to operate in ways it was never intended to do so. That's my two cents on the topic. HTH
>

[Michael]

In my opinion, decorate the base-est whatever it is, be it class, method, property, etc, and let the language(s) (as the case may be) work *for* you, instead of fighting what's there. That includes things like attribution. If it made sense to allow for multiple, inherited use cases, then by all means. It's a delta of GetCustomAttributes. Indeed, sometimes that comes in the form of test fixture hierarchy, sometimes I have extended an object under test to expose features into test that would ordinarily be hidden from view in production, and have also used extensions; by no means is that exhaustive in cooperation with the test framework, the parts are there for creative use. That's my two cents. HTH

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Paul Hicks]

I've rejigged my expectations and achieved a more-than-adequate outcome. Borrowing from your example on the wiki, I've changed my ActionAttribute's Targets to ActionTargets.Suite | ActionTargets.Test, and put my clean-slate code into BeforeTest (if ITest.IsSuite) and AfterTest (if !ITest.IsSuite). End result is that I clean up before starting my suite (to recover from past error conditions, weird states caused by direct DB access, etc.) and after each test. This achieves everything needed in my current cases.

[Michael Powell]

Nice. Glad I could help.

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "NUnit-Discuss" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nunit-discuss/xUA5hMEYUgc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to