I am wondering what the best method for conditionally ignoring a test is

[Pat O]

Hello,
We test our software with NUnit. Our software communicates to our
hardware. We have many different types of hardware but they fall into
3 or 4 groups of functionality. Some of our unit tests only work with
one type of functionality. So lets say that hardware group A can do
math. Our unit tests that test the math functionality should only be
run on hardware in group A.
We make a list of all the hardware that we might be able to test
on. Then the various unit tests machines "check out" hardware they
are going to use for testing. Then all the unit tests are run on all
the hardware. We have created "special" error messages to know that a
test faled because it was run on a peice of hardware that does not
support the functionality. Is there a method that I can use that is
part of NUnit to tell the framework to ignore the test? Remember this
has to happen at runtime after we have connected to the hardware.

Thanks for any help.
Pat O

[Charlie Poole]

Hi Pat,

If you were able to know which hardware was to be used by a run, the
easiest approach would be to use Categories on the tests. Then the
run on a particular test server could include only the categories that
apply to the hardware it was going to check out.

However, it sounds like you need to do it after the run starts, with
the tests themselves detecting what's available. In that situation,
you can use one of several runtime actions:

1) Simply not executing the test code or using Assert.Pass, which
will make the test succeed. This is a bit of a misrepresentation,
since you haven't actually run the test. Assert.Pass does have
a message associated with it but it's not currently reported
in any useful way (like in the XML) so that won't help you.

2) Make an assertion, causing the test to fail. I guess this is
what you do now, with the disadvantage that you have to filter
the "failures" manually.

3) Use Assert.Ignore, causing it to be ignored. This would give
you a yellow bar in the gui and a list of ignored tests in the
console runner. You should still go through the list, in order
to make sure that there are not other ignored tests that need
to be dealt with.

4) Use Assert.Inconclusive directly or better yet Assume.That to
give you an inconclusive result. This does not affect the overall
outcome of the run and does not clutter your results with messages
so it seems like a likely candidate right now. The test in your
case isn't truly inconclusive but more like "undetermined" but
that seems like a minor point. We may end up reporting inonclusive
tests at a later point, but probably only as an option.

5) It seems to me that the ideal (but not yet available) solution
for your case would be to allow you to extend the platform attribute
to specify your own hardware together with your own detection
routine. Then it would work just like a test that is intended to
run only on a specific operating system. Unfortunately, we don't
have that yet.

So for now, I suggest #4. You could code it in the test itself
or, where applicable, in the SetUp or TestFixtureSetUp as
Assume.That( RequiredHardwareIsAvailable(), "some message");

Charlie