Custom Attributes

[Ed Blackburn]

Hi, I tried to post earlier but it doesn't seem to have appeared on the board? Anyway...

We currently have a suite of acceptance tests using NUnit and Selenium / Web Driver 2. We'd like to introduce some behaviour that in NUnit terms means extending the runner. This seems expensive because, NUnit3 (aka Duke Nukem Forever .NET) includes an easier extension model. NUnits IAddin hosting model is guarnteed to work with TeamCity or R#.

As we're already using MSpec for unit testing, I thought I'd look at how we may achieve our goals in MSpec. Initially our first two requirements are:

• Conditional Attribute
  

Our tests are environment aware because different environments have some features in different states. As such it'd be nice to express this like so: [FeatureRequiredAttribute("NewFeatureX")] which ignores the test if the target environment doesn't have that feature on. 

• TryAgain Attribute
  

Some of our more brittle tests are more intermittent, predominantly because of environment constraints. As such we'd like to be able to stipulate a max number of times a test is attempted. i.e. [TryAgainAttribute(3)]

Does anyone know if this achievable or desirable? If not how could one go about extending the runner? Could the runner be extended to scan the assemblies under test to compose the extended points as opposed to NUnits convention in the runner hosted filesystem?

Suggestions most welcome?

Thanks, Ed

[Alexander Groß]

Hi Ed,

Ed Blackburn wrote on 28/11/2011 at 14:58:01:
> * Conditional Attribute

> Our tests are environment aware because different environments
> have some features in different states. As such it'd be nice to express
> this like so: [FeatureRequiredAttribute("NewFeatureX")] which ignores
> the test if the target environment doesn't have that feature on.

You can use [TagsAttribute("NewFeatureX")] above your contexts and limit
what's run using the console runner:

mspec.exe -include NewFeatureX SomeAssembly.dll

> * TryAgain Attribute Some of our more brittle tests are more

> intermittent, predominantly because of environment constraints. As such
> we'd like to be able to stipulate a max number of times a test is
> attempted. i.e. [TryAgainAttribute(3)]

Mh, I would argue you should fix your brittle tests first. MSpec doesn’t
have the notion of retries, hence there's no attribute for it. To support
retries I'm afraid you would have to write a new DefaultRunner and hook it
up in code. Currently, the only way to get notified about specification
results is to implement ISpecificationListener, but as the name states it's
only able to listen, but not to act.

> If not how could one
> go about extending the runner?

As you probably guessed, MSpec currently doesn't provide infrastructure to
extend the runner. You're welcome to discuss what's needed for your use
cases and how we would go about implementing the features. I'm not really
fond of loading assemblies/addins. Personally, I would prefer to have the
developer implement an interface that might change specifics about the test
run as we currently do it with IAssemblyContext and
ICleanupAfterEveryContextInAssembly.

HTH,

Alex
--
Alexander Groß
http://therightstuff.de/