Disable warnings

[Joe Enos]

Is there any way to disable specific warnings from NUnit? My scenario
is as follows:

I have a generic base class that serves as my test fixture. It's an
abstract class, so it's something like FooTestFixture<T>. All of my
tests are inside of this class.

I have two (or more) concrete classes that derive from
FooTestFixture<RealClass>, and only implement a few methods that were
abstract in the base. Each of these concrete classes run all of the
tests from the base class when NUnit runs.

The concrete classes work fine, and NUnit reports overall success, but
my problem is that NUnit gives me warnings about the abstract class,
which uglies up the output. The warnings are "FooTestFixture`1 is an
abstract class".

Is there any way to disable these warnings?

I looked for alternative ways of doing these tests other than abstract
classes, but there really aren't any good ways - I'd have to duplicate
code, or run unnecessary tests.

Thanks for the help.

[Charlie Poole]

Hi Joe,

The standard way to avoid the warnings is to not put a TestFixtureAttribute
on the abstract class. Put it on the concrete classes instead.

That said, you may have some special case here based on an interaction
between the use of generics and the abstract class, so if the above
solution doesn't work please post a bit more info.

Charlie

> --
>
> You received this message because you are subscribed to the
> Google Groups "NUnit-Discuss" group.
> To post to this group, send email to nunit-...@googlegroups.com.
> To unsubscribe from this group, send email to
> nunit-discus...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/nunit-discuss?hl=.
>
>
>

[Joe Enos]

All of my test fixtures derive from an abstract class that I call
TestFixtureBase, and the [TestFixture] attribute is only on that base
class - this lets me avoid having to use that attribute on all of my
concrete classes, in addition to all the base functionality that the
base class contains for setup, teardown, etc.

So my FooTestFixture<T> abstract class derives from TestFixtureBase,
which has the attribute - so I can't avoid this, unless I removed the
attribute from TestFixtureBase, and added it to all of my concrete
classes.

NUnit doesn't seem to care about the TestFixtureBase abstract class,
because there aren't any tests in it.

Thanks for the idea, but it won't work in my case.

[Charlie Poole]

Hi Joe,

> All of my test fixtures derive from an abstract class that I
> call TestFixtureBase, and the [TestFixture] attribute is only
> on that base class - this lets me avoid having to use that
> attribute on all of my concrete classes, in addition to all
> the base functionality that the base class contains for
> setup, teardown, etc.
>
> So my FooTestFixture<T> abstract class derives from
> TestFixtureBase, which has the attribute - so I can't avoid
> this, unless I removed the attribute from TestFixtureBase,
> and added it to all of my concrete classes.
>
> NUnit doesn't seem to care about the TestFixtureBase abstract
> class, because there aren't any tests in it.
>
> Thanks for the idea, but it won't work in my case.

NUnit is designed such that the TestFixtureAttribute has to
be on the concrete classes, so putting it on an abstract base
will make it impossible to ever get a green bar.

You could put the attribute on a (non-abstract) generic class
without any problem.

Having NUnit simply disregard TestFixtureAttribute on an abstract
class would solve your problem but would silently ignore the
situation where the abstract class was never used by any
concrete class. One could argue that this would be a feature,
or a bug, depending on point of view.

I'm open to suggestions on this one.

Charlie

[Joe Enos]

NUnit does recognize the concrete classes even without the attribute -
it must look at the entire class hierarchy to determine which classes
are test fixtures. The GUI app, console app, integration with Cruise
Control, Resharper, and SharpDevelop all recognize them, so it looks
like NUnit was meant to allow this.

I really can't think of a good way to implement it - only thing I can
think of off the top of my head would be if NUnit had an attribute
that you could put in an intermediary class like mine, whose purpose
is to say "This particular class should not be considered a test
fixture, but any children should be" - let's call it
DisableAbstractWarningAttribute for now. Then I could have:

[TestFixture]
public abstract class TestFixtureBase { ... }

[DisableAbstractWarning]
public abstract class FooTestFixture<T> : TestFixtureBase {
[Test]
public void Test1() {
T t = GetMyT();
TestSomething(t);
}
protected abstract T GetMyT();
private void TestSomething(T t) {
// Implement
}
}

public class RealTestFixture : FooTestFixture<Foo> {
protected override Foo GetMyT() { ... }
}

public class AnotherRealTestFixture : FooTestFixture<Bar> {
protected override Bar GetMyT() { ... }
}

public class Foo { }
public class Bar { }

This is pretty much exactly how I have my classes now, except without
the [DisableAbstractWarning] attribute. The Test1 test fires for all
concrete classes, but I get a warning telling me that FooTestFixture`1
is abstract - it's a warning only, so the overall test doesn't fail.
A DisableAbstractWarningAttribute would allow me to indicate that I
know what I'm doing - that I know this is abstract and will never fire
on its own, but I want to keep it anyway.

Seems kind of cheesy, but it's all I can think of right now, other
than a global setting (app.config maybe) that turns off the warning
entirely. Other than that, I guess I just have to live with the
warnings, and try to avoid this pattern whenever I can, to keep the
problem from growing.

Thanks for your help.

Joe

[Joe Enos]

Actually, I take that back - SharpDevelop doesn't recognize them.

[Charlie Poole]

Hi Joe,

> NUnit does recognize the concrete classes even without the
> attribute - it must look at the entire class hierarchy to
> determine which classes are test fixtures. The GUI app,
> console app, integration with Cruise Control, Resharper, and
> SharpDevelop all recognize them, so it looks like NUnit was
> meant to allow this.

Yes, it does. Back in early 2.0 days this feature was used
to allow you to inherit from NUnit.Framework.TestCase as
was done in NUnit 1.x. That was a non-abstract class with
SetUp and TearDown methods but no tests. You can still use
the inherited attribute successfully that way.

However, if you add tests to the base class, those tests
will be executed by an instance of the base class. And if
you make the test class abstract, you get a warning because
NUnit can't instantiate it.

FWIW, however, you can't rely on what 3rd party products
do to indicate what NUnit was designed to do. Resharper is
notorious in this way since it doesn't actually run tests
with NUnit. Basically if a 3rd party app runs NUnit using
NUnit's own runners, then it works like NUnit. If it uses
it's own code, it works the way that code is written, which
may or may not match NUnit.

It's an option I've considered and - yes - it does seem
cheesy. :-) Another thought is to have an AbstractFixture
attribute, which only takes effect when inherited by a
non-abstract class. That's a bit trickier to implement
but seems like a better approach for a future release.

Does anyone else have some ideas?

Charlie

[Charlie Poole]

Most likely, the addin doesn't use NUnit to run tests. Look for
references to nunit.core and nunit.util to know for sure.

[Joe Enos]

Thanks for the insight - I guess "kind of works ok" and "is made for
it" are two different things. I suppose for now I'll rethink my
design pattern and try to get my tests outside of the base class. If
nothing else, I can have the methods inside of the base class without
the TestAttribute, then each class will have corresponding methods
that just point back to the base, like:

public class FooTextFixtureBase<T> {
protected /* virtual? */ void Test01() { /* actually test stuff */ }
}
public class RealTextFixture : FooTestFixtureBase<Foo> {
[Test]
public /* new or override */ void Test01() { base.Test01(); }
}

But if my original pattern is something that would be useful to
people, it might be worth putting in a feature request. Let me know
if you think so, and I'll put it through the proper channels.

Thanks for your help. I appreciate the opportunity to talk with
someone who actually is developing the product.

Joe

[Charlie Poole]

Hi All,

Let's talk about this as a feature proposal.

Suppose we had an AbstractTestFixtureAttribute.

It could derive from TestFixtureAttribute and use all
the same constructors. The only diff would be that it
would cause the class on which it was placed to *never*
be used as a test fixture, even if that class contained
tests.

Similarly, it would be ignored on abstract descendant
classes.

On a non-abstract derived class, it would function exactly
like the existing TestFixtureAttribute.

See the six example use cases below, especially #5. Would
that one be a problem? Is it sufficient that any descendant
not intended to be instantiated by NUnit must be abstract?

Charlie

Case 1:

[AbstractTestFixture]
abstract class Base
{
// Would not be instantiated by NUnit
}

class Derived : Base
{
// Would be instantiated by NUnit
}

Case 2:

[AbstractTestFixture]
class Base // Note this is not abstract
{
// Would not be instantiated by NUnit
}

class Derived : Base
{
// Would be instantiated by NUnit
}

Case 3:

[AbstractTestFixture]
abstract class Base
{
// Would not be instantiated by NUnit
}

abstract class Derived : Base
{
// Would not be instantiated by NUnit
}

class Concrete : Derived
{
// Would be instantiated by NUnit
}

Case 4:

[AbstractTestFixture]
class Base // Not abstract
{
// Would not be instantiated by NUnit
}

abstract class Derived : Base
{
// Would not be instantiated by NUnit
}

class Concrete : Derived
{
// Would not be instantiated by NUnit
}

Case 5 (potential problem):

[AbstractTestFixture]
abstract class Base
{
// Would not be instantiated by NUnit
}

class Derived : Base
{
// Would not be instantiated by NUnit,
// which might not be intended.
}

class Concrete : Derived
{
// Would not be instantiated by NUnit
}

Case 6:

// Class assumed to be part of an external framework
// for which source is not available.
class External
{
// Would not be instantiated by NUnit
}

[AbstractTestFixture]
abstract class Base : External
{
// Would not be instantiated by NUnit
}

abstract class Derived : Base
{
// Would not be instantiated by NUnit
}

class Concrete : Derived
{
// Would not be instantiated by NUnit

}

> -----Original Message-----
> From: Joe Enos [mailto:j...@jtenos.com]

> Sent: Wednesday, November 18, 2009 12:32 PM
> To: NUnit-Discuss
> Subject: [nunit-discuss] Re: Disable warnings
>

[Joe Enos]

Your logic sounds good to me. For #5, I don't think it's reasonable
to walk the entire tree of objects to look for any descendants, and
skip tests for any class that has children. I think your assumption
that AbstractTestFixtureAttribute means "this class and all *abstract*
children" is good. In the #5 case, I'd suggest that both "Derived"
and "Concrete" would be instantiated, and all tests run for both. In
my opinion, if you wanted "Derived" to not be instantiated and run,
then the developer should slap another AbstractTestFixtureAttribute
onto it - or simply make it an abstract class.

[Kenneth Xu]

I don't understand the problem.

I use abstract/reusable test fixtures a lot. With 2.5+, you don't need
to specify the TestFixtureAttribute at all unless you need to pass in
parameters, which doesn't make sense for abstract test fixtures.

Please correct me if I'm wrong but I think all Joe has to do is simply
remove the attribute from the abstract base class and that's it.

Cheers,
Kenneth

[Charlie Poole]

Hi Kenneth,

"As a user" of NUnit, I don't see the problem either.

But I've been hearing about it as a problem from others for
some time. It's particularly a problem in use case #5.
Consider this...

1) You derive all your test fixtures from an abstract
base class.

2) That base is in turn derived from a third-party
base class, which already provides the TestFixture
attribute.

Notice that in this case the only current way to
avoid the yellow bar is to modify the third-party
code, which could be undesirable for other reasons
and is sometimes impossible. Believe it or not,
this has actually come up a few times.

So my inclination is to simply remove the problem,
even though the real bug is (IMO) with the 3rd
party code that uses the attribute.

Charlie

[Kenneth Xu]

Hi Charlie,

> 1) You derive all your test fixtures from an abstract
> base class.
>
> 2) That base is in turn derived from a third-party
> base class, which already provides the TestFixture
> attribute.
>
> Notice that in this case the only current way to
> avoid the yellow bar is to modify the third-party
> code, which could be undesirable for other reasons
> and is sometimes impossible. Believe it or not,
> this has actually come up a few times.

I couldn't reproduce this. Am I missing something here? As long as I
don't put the TestFixture attribute on any of abstract classes that I
own, I don't get any warning. The 3rd party class is in a different
assembly and it won't be picked up by the NUnit in the first place.

Joe,

If you remove the TestFixture from all the abstract classes that you
own, do you still get any warning?

Cheers,
Kenneth

[Joe Enos]

Kenneth:
The warnings only come up if there are tests in the abstract class.
If the abstract class only contains setup, teardown, and other non-
test code, then the NUnit GUI gives me a yellow on the class, but
NUnit doesn't report a warning.

The warning doesn't appear if the TestFixtureAttribute is removed from
the abstract class, but as Charlie mentioned, it's not always possible
to remove it from the base class, if it's third-party. Or in my case,
I can remove it from the base, but that would require me to add it to
every class that derives from that base (not really a problem, but not
what I'm currently doing).

[Kenneth Xu]

Hi Joe,

> it's not always possible
> to remove it from the base class, if it's third-party.

As I mentioned in my email earlier, if it is a third-party, it's not
in your test assembly so it won't get picked up by NUnit in the first
place. Hence there is no warning.

> Or in my case,
> I can remove it from the base, but that would require me to add it to
> every class that derives from that base (not really a problem, but not
> what I'm currently doing).

Well, you don't need to added the TestFixture to your derived classes.
NUnit 2.5 should be able to pick up your tests just fine without
TestFixture attribute.

Cheers,
Kenneth

[Joe Enos]

I see your point about the 3rd party library - sorry, I misunderstood
the first time you mentioned it. Just tried that and it worked
correctly, even with no TestFixtureAttribute on my concrete class.

Kind of a cool feature for NUnit 2.5 to not require
TestFixtureAttribute - We're still on 2.4.3, so I can't take advantage
of that yet, but I'm sure we'll move up one of these days.

While I was checking out some of the new 2.5 stuff (
http://nunit.com/index.php?p=testFixture&r=2.5 ) I found something
else pretty interesting - the generic test fixtures, where a single
test fixture class can be run multiple times, with different types,
like:

[TestFixture(typeof(Foo))]
[TestFixture(typeof(Bar))]
public class MyTestFixture<T> where T : Foo { }

public class Foo { }
public class Bar : Foo { }

I could then put all of the tests into MyTestFixture, and put the
specific implementations of the important stuff into Foo and Bar.
This would take a little bit of reorganizing from where I'm at today,
but it would work. I'd still need attributes for every concrete type
that uses this feature, but I think I can live with that.

Thanks to both of you guys for your help.

[Kenneth Xu]

> I'd still need attributes for every concrete type
> that uses this feature, but I think I can live with that.

Yes, that's what I meant by "unless you need to pass in parameters",
in this case it's a type parameter. Just think this is a way to pass
parameter to your test fixture instead of marking it to be a test
fixture.

> Thanks to both of you guys for your help.

My pleasure.

[Charlie Poole]

Hi Kenneth,

> > it's not always possible
> > to remove it from the base class, if it's third-party.
>
> As I mentioned in my email earlier, if it is a third-party,
> it's not in your test assembly so it won't get picked up by
> NUnit in the first place. Hence there is no warning.

If you inherit from the third-party class, the attribute
is visible on your own classes.

> > Or in my case,
> > I can remove it from the base, but that would require me to
> add it to
> > every class that derives from that base (not really a
> problem, but not
> > what I'm currently doing).
>
> Well, you don't need to added the TestFixture to your derived classes.
> NUnit 2.5 should be able to pick up your tests just fine
> without TestFixture attribute.

Not all tests - only those marked by Test, TestCase, TestCaseSource
or Theory. A broader interpretation would be possible, of course.

Charlie

[Charlie Poole]

Hi Joe,

Just a side note... my earlier responses were all colored to
some degree by the existence of both parameterized and generic
test fixtures and may make more sense now that you've seen
those two features.

Charlie

> -----Original Message-----
> From: Joe Enos [mailto:j...@jtenos.com]
> Sent: Friday, November 20, 2009 11:53 AM
> To: NUnit-Discuss

[Joe Enos]

Yes, things are a lot clearer now. And I just found out that we're
able to switch to 2.5, so I'll be able to do this stuff sooner than I
thought.

Thanks again for your help.

On Nov 20, 1:45 pm, "Charlie Poole" <char...@nunit.com> wrote:
> Hi Joe,
>
> Just a side note... my earlier responses were all colored to
> some degree by the existence of both parameterized and generic
> test fixtures and may make more sense now that you've seen
> those two features.
>
> Charlie
>
> > -----Original Message-----
> > From: Joe Enos [mailto:j...@jtenos.com]
> > Sent: Friday, November 20, 2009 11:53 AM
> > To: NUnit-Discuss
> > Subject: [nunit-discuss] Re: Disable warnings
>
> > I see your point about the 3rd party library - sorry, I
> > misunderstood the first time you mentioned it.  Just tried
> > that and it worked correctly, even with no
> > TestFixtureAttribute on my concrete class.
>
> > Kind of a cool feature for NUnit 2.5 to not require
> > TestFixtureAttribute - We're still on 2.4.3, so I can't take
> > advantage of that yet, but I'm sure we'll move up one of these days.
>
> > While I was checking out some of the new 2.5 stuff (

> >http://nunit.com/index.php?p=testFixture&r=2.5) I found

[Kenneth Xu]

Hi Charlie,

>> As I mentioned in my email earlier, if it is a third-party,
>> it's not in your test assembly so it won't get picked up by
>> NUnit in the first place. Hence there is no warning.
>
> If you inherit from the third-party class, the attribute
> is visible on your own classes.

When I tested, I don't get warning even if I have abstract test class
inherit from 3rd party test class with TestFixture attribute as long
as I don't put on my own abstract test class. What you said is
theoretically correct only if you indicate including inherited when
you query for attribute. Didn't check the code but I guess NUnit
doesn't query inherited attributes.

>> Well, you don't need to added the TestFixture to your derived classes.
>> NUnit 2.5 should be able to pick up your tests just fine
>> without TestFixture attribute.
>
> Not all tests - only those marked by Test, TestCase, TestCaseSource
> or Theory. A broader interpretation would be possible, of course.

That's all I know about :p. Another is method name end with Test,
which I don't even know if still works in NUnit. What are others? :)

Thanks,
Kenneth

[Charlie Poole]

Hi Kenneth,

> >> As I mentioned in my email earlier, if it is a
> third-party, it's not
> >> in your test assembly so it won't get picked up by NUnit
> in the first
> >> place. Hence there is no warning.
> >
> > If you inherit from the third-party class, the attribute is
> visible on
> > your own classes.
>
> When I tested, I don't get warning even if I have abstract
> test class inherit from 3rd party test class with TestFixture
> attribute as long as I don't put on my own abstract test
> class. What you said is theoretically correct only if you
> indicate including inherited when you query for attribute.
> Didn't check the code but I guess NUnit doesn't query
> inherited attributes.

NUnit definitely queries for inherited TestFixture attributes.
If you have an example where it doesn't, you may have found a bug.

> >> Well, you don't need to added the TestFixture to your
> derived classes.
> >> NUnit 2.5 should be able to pick up your tests just fine without
> >> TestFixture attribute.
> >
> > Not all tests - only those marked by Test, TestCase,
> TestCaseSource or
> > Theory. A broader interpretation would be possible, of course.
>
> That's all I know about :p. Another is method name end with
> Test, which I don't even know if still works in NUnit. What
> are others? :)

NUnit's own data providers on the individual parameters do not
cause a test fixture to be recognized - but they don't cause
the method to be recognized as a test either, so I guess that's
a no-brainer.

But the main exception is any attributes used by TestCaseBuilder
or DataProvider addins - unless the addin author also provides
a SuiteBuilder extension.

We'll probably iron out these irregularities in 3.0.

Charlie

> Thanks,

[Kenneth Xu]

Hi Charlie Poole,

> NUnit definitely queries for inherited TestFixture attributes.
> If you have an example where it doesn't, you may have found a bug.

It's quite easy to reproduce. You can follow what I described below or
get them here:
svn co http://kennethxublogsource.googlecode.com/svn/trunk/Sandbox Sandbox

Create two projects in a solution and add below classes:

NUnitStuff project:

[TestFixture]
public abstract class ThirdPartyFixture
{
[Test] public void ThirdPartyTest() {}
}

NUnitStuff.Tests project:

public abstract class AbstractFixture
{
[Test] public void TestOnAbstractFixture() { }
[Test] public virtual void OverrideableTestOnAbstractFixture() { }
}
public class DerivedFixture : AbstractFixture
{
[Test] public void TestOnDerivedFixture() { }
public override void OverrideableTestOnAbstractFixture() { }
}

Compile and use NUnit to run the NUnitStuff.Tests project, there are
two evidences:

1. I don't get warning for test in the AbstractFixture.
2. NUnit didn't pick up test case: OverrideableTestOnAbstractFixture

> NUnit's own data providers on the individual parameters do not
> cause a test fixture to be recognized - but they don't cause
> the method to be recognized as a test either, so I guess that's
> a no-brainer.
>
> But the main exception is any attributes used by TestCaseBuilder
> or DataProvider addins - unless the addin author also provides
> a SuiteBuilder extension.

I never wrote addin but this is good to know, thanks much.

Cheers,
Kenneth

[Charlie Poole]

Hi Kenneth,

I'll be on planes most of today, so I'll try to answer just based
on the email and do some experimenting tomorrow.

> > NUnit definitely queries for inherited TestFixture attributes.
> > If you have an example where it doesn't, you may have found a bug.
>
> It's quite easy to reproduce. You can follow what I described
> below or get them here:
> svn co
> http://kennethxublogsource.googlecode.com/svn/trunk/Sandbox Sandbox
>
> Create two projects in a solution and add below classes:
>
> NUnitStuff project:
>
> [TestFixture]
> public abstract class ThirdPartyFixture
> {
> [Test] public void ThirdPartyTest() {}
> }
>
> NUnitStuff.Tests project:
>
> public abstract class AbstractFixture
> {
> [Test] public void TestOnAbstractFixture() { }
> [Test] public virtual void
> OverrideableTestOnAbstractFixture() { }
> }
> public class DerivedFixture : AbstractFixture
> {
> [Test] public void TestOnDerivedFixture() { }
> public override void OverrideableTestOnAbstractFixture() { }
> }
>
> Compile and use NUnit to run the NUnitStuff.Tests project,
> there are two evidences:
>
> 1. I don't get warning for test in the AbstractFixture.

I wouldn't expect it to. It's not in the assembly that NUnit
is scanning for tests. Separating such things into a different
assembly is a good strategy for avoiding the warning message.

> 2. NUnit didn't pick up test case: OverrideableTestOnAbstractFixture

I missed that we were talking about an overridden method. I'm
pretty sure NUnit does not have a test case for this.

Try putting a non-abstract test case in the base to see what
happens.

NUnit looks for inherited attributes for TestFixture, SetUp,
TearDown, TestFixtureSetUp and TestFixtureTearDown but *not*
for Test, TestCase, etc. Here's the reasoning:

TestFixture needs to have this to support inherited fixtures.

SetUp et al. is a historical thing: we used this approach
to allow easy transitioning from NUnit 1.11 to 2.0. It's
probably no longer needed - unless, of course, people have
discovered it and are using it for something else!

Tests have always been considered as invariant. Whether they
are specified in a base class or not is irrelevant - they
run because they are methods of the class being scanned
by NUnit and they always have the attribute defined where
the tests are defined.

The "usual" pattern is to use a template pattern for inherited
setup or test that needs to be tailored in the derived fixture.

I quoted "usual" because it's what I think is usual - I could
be wrong about how people are using this! However, it is the
pattern that currently works. :-)

Let's separate the problem we were discussing (inheritance
of abstract fixures) from this new one (overriding tests) and
see if we need a new feature. Obviously, it's doable if there
is a real use case for it but I wouldn't want to make things
more complicated without a reason.

Charlie

> > NUnit's own data providers on the individual parameters do
> not cause a
> > test fixture to be recognized - but they don't cause the
> method to be
> > recognized as a test either, so I guess that's a no-brainer.
> >
> > But the main exception is any attributes used by TestCaseBuilder or
> > DataProvider addins - unless the addin author also provides a
> > SuiteBuilder extension.
>
> I never wrote addin but this is good to know, thanks much.


> Cheers,
> Kenneth
>

[Kenneth Xu]

Hi Charlie,

>> NUnitStuff project:
>>
>>     [TestFixture]
>>     public abstract class ThirdPartyFixture
>>     {
>>         [Test] public void ThirdPartyTest() {}
>>     }
>>
>> NUnitStuff.Tests project:
>>
>>     public abstract class AbstractFixture
>>     {
>>         [Test] public void TestOnAbstractFixture() { }
>>         [Test] public virtual void
>> OverrideableTestOnAbstractFixture() { }
>>     }
>>     public class DerivedFixture : AbstractFixture
>>     {
>>         [Test] public void TestOnDerivedFixture() { }
>>         public override void OverrideableTestOnAbstractFixture() { }
>>     }
>>
>> Compile and use NUnit to run the NUnitStuff.Tests project,
>> there are two evidences:
>>
>> 1. I don't get warning for test in the AbstractFixture.
>
> I wouldn't expect it to. It's not in the assembly that NUnit
> is scanning for tests. Separating such things into a different
> assembly is a good strategy for avoiding the warning message.

The example I used is exactly as what you have suggested. While the
ThirdPartyFixture, which HAS the TestFixtureAttribute, is not in the
assembly that NUnit is scanning the tests, but the AbstractFixture IS
in the scanned assembly, without TestFixtureAttribute directly
specified but inherited from ThirdPartyFixture. According to what you
have suggested earlier, I will get a warning, so you were introducing
another attribute to avoid it. But according to above test, I didn't
get any warning.

I'll leave the Test aside for now maybe continue in another thread later.

Cheers,
Kenneth

[Kenneth Xu]

Hi Charlie,

Please ignore my previous email. I made a mistake in my code and yes,
you are right, the TestFixtureAttribute IS inherited.

I apologize for any confusion caused.

Thanks,
Kenneth.

[Charlie Poole]

Hi Kenneth,

> The example I used is exactly as what you have suggested.
> While the ThirdPartyFixture, which HAS the
> TestFixtureAttribute, is not in the assembly that NUnit is
> scanning the tests, but the AbstractFixture IS in the scanned
> assembly, without TestFixtureAttribute directly specified but
> inherited from ThirdPartyFixture. According to what you have
> suggested earlier, I will get a warning, so you were
> introducing another attribute to avoid it. But according to
> above test, I didn't get any warning.

Yes, this should a problem, not on the ThirdPartyFixture, but
on the AbstractFixture. I'm not clear why you're not seeing it
but I'll look more closely tomorrow.

Charlie

> I'll leave the Test aside for now maybe continue in another
> thread later.
>
> Cheers,
> Kenneth
>

[Charlie Poole]

Hi Kenneth,

> Please ignore my previous email. I made a mistake in my code
> and yes, you are right, the TestFixtureAttribute IS inherited.

OK, noted.

> I apologize for any confusion caused.

No sweat.

Charlie

[Kenneth Xu]

Hi Charlie,

Thanks for your patience. With all that being sorted out and clear,
I'm now wondering what are invalid tests except below listed:

1. Tests in abstract class with TestFixtureAttribute
2. Parameterized tests with no arguments provided
3. Test fixture with no matching constructor

Also, given another thought, it seems to be odd to me that making a
class abstract is not enough to express this is an abstract test
fixture to be inherited by other concrete test fixture, but I'll need
another AbstractFixture attribute to double assert it. :)

Another interesting fact is that if I forgot to provide type parameter
to generic test fixture, I don't get any warning. Those test cases
would truly never be executed.

// forgot [TestFixture(typeof(string))]
public class GenericTestFixture<T> : ThirdPartyFixture
{
[Test] public void TestOnGenericFixture()
{
}
}

Cheers,
Kenneth

[Charlie Poole]

Hi Kenneth,

> Thanks for your patience. With all that being sorted out and
> clear, I'm now wondering what are invalid tests except below listed:
>
> 1. Tests in abstract class with TestFixtureAttribute 2.
> Parameterized tests with no arguments provided 3. Test
> fixture with no matching constructor

From a quick glance at code, I come up with these reasons
for a test fixture to be invalid:

1. Class is abstract
2. Class generic without proper type parameters provided
3. Class has a SetUp, TearDown, TestFixtureSetUp or TestFixtureTearDown
method with an invalid signature.
4. Class has no appropriate constructor (i.e. one that matches the
arguments provided or default constructor if none were provided)

Individual test methods may be invalid for these reasons:

1. Method is abstract
2. Method is not public
3. Method needs args and none are provided
4. Method takes no args but args are provided
5. Provided args don't match the method signature
6. Method returns a value but no expected result was provided
7. Method is generic and type args cannot be deduced from arguments
8. An error occurs in building test cases

> Also, given another thought, it seems to be odd to me that
> making a class abstract is not enough to express this is an
> abstract test fixture to be inherited by other concrete test
> fixture, but I'll need another AbstractFixture attribute to
> double assert it. :)

Reasons:
1. History: We have always treated marking a fixture class
abstract as an error and NUnit tends to avoid silently passing
by possible errors. It may be that we don't want to treat it
as an error any longer, since there are useful patterns that
require it.

2. Philosophy: NUnit has to work with all .NET languages,
whether they have an abstract keyword or not. So if we
want to support this pattern in such languages, we need
some attribute or property to help us out.

I'm comfortable with a change that simply ignores TestFixture
on an abstract class, but it's a big change and we need to give
people whose usage patterns might be disrupted a chance to
voice any objections.

> Another interesting fact is that if I forgot to provide type
> parameter to generic test fixture, I don't get any warning.
> Those test cases would truly never be executed.
>
> // forgot [TestFixture(typeof(string))]
> public class GenericTestFixture<T> : ThirdPartyFixture
> {
> [Test] public void TestOnGenericFixture()
> {
> }
> }

This looks like a bug to me, assuming that ThirdPartyFixture
has a TestFixture attribute on it.

[Kenneth Xu]

Hi Charlie,

> From a quick glance at code, I come up with these reasons
> for a test fixture to be invalid:
>
> 1. Class is abstract
> 2. Class generic without proper type parameters provided
> 3. Class has a SetUp, TearDown, TestFixtureSetUp or TestFixtureTearDown
> method with an invalid signature.
> 4. Class has no appropriate constructor (i.e. one that matches the
> arguments provided or default constructor if none were provided)
>
> Individual test methods may be invalid for these reasons:
>
> 1. Method is abstract
> 2. Method is not public
> 3. Method needs args and none are provided
> 4. Method takes no args but args are provided
> 5. Provided args don't match the method signature
> 6. Method returns a value but no expected result was provided
> 7. Method is generic and type args cannot be deduced from arguments
> 8. An error occurs in building test cases

Nice list, this is good doc material indeed.

> 2. Philosophy: NUnit has to work with all .NET languages,
> whether they have an abstract keyword or not. So if we
> want to support this pattern in such languages, we need
> some attribute or property to help us out.

Didn't though about this. Yes, the attribute is useful in this case.

> I'm comfortable with a change that simply ignores TestFixture
> on an abstract class, but it's a big change and we need to give
> people whose usage patterns might be disrupted a chance to
> voice any objections.

Sounds good to me.

>>     // forgot [TestFixture(typeof(string))]
>>     public class GenericTestFixture<T> : ThirdPartyFixture
>>     {
>>         [Test] public void TestOnGenericFixture()
>>         {
>>         }
>>     }
>
> This looks like a bug to me, assuming that ThirdPartyFixture
> has a TestFixture attribute on it.

Yes, ThirdPartyFixture has the TestFixture attribute on it. But I
think NUnit should flag invalid even when it doesn't inherit from
ThirdPartyFixture. Shall I create an issue for this?

Cheers,
Kenneth

[Charlie Poole]

Hi Kenneth,

> Yes, ThirdPartyFixture has the TestFixture attribute on it.
> But I think NUnit should flag invalid even when it doesn't
> inherit from ThirdPartyFixture. Shall I create an issue for this?

Please do. It's a framework issue, so it applies to both 2.5 and
3.0. You can just file it on one and then add the other as an affected
project.

Charlie

[Kenneth Xu]

I created the bug reports.

Cheers,
Kenneth

> --
>
> You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
> To post to this group, send email to nunit-...@googlegroups.com.
> To unsubscribe from this group, send email to nunit-discus...@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/nunit-discuss?hl=en.
>
>
>