running with nunit-console AfterTestRun is never triggered

[Kevin]

Hi,

I am running my tests with nunit-console and after tests finish
running I would expect the event AfterTestRun to be triggered so I can
stop my dev server (WebDev.WebServer.exe) and dispose other
resources.

But when tests finish AfterTestRun doesn't get hit so my server
remains started, and if I want to run my tests again, the server will
fail to start because the port is already in use.

Why doesn't AfterTestRun get hit? I look into the list of processes in
Task Manager and I don't see nunit there.

[John Gietzen]

This was a known bug with NUnit 2.5.8. If you upgrade to 2.5.9, this
should be fixed.

Sent from my Windows Phone From: Kevin
Sent: Wednesday, January 05, 2011 11:01 PM
To: SpecFlow
Subject: [specflow] running with nunit-console AfterTestRun is never
triggered

[Kevin]

Nope, this didn't work.

When I run NUnit from within Visual Studio setting it as an external
tool using the NUnit Gui Runner, the AfterTestRun gets hit only when I
close the NUnit Gui Runner.
But if I run the tests using the nunit-console, the AfterTestRun
doesn't seem to ever get hit, since when I close the console, the
WebDev.WebServer.exe is still there.

I see the same behavior with the former 2.5.7 version of NUnit I was
using and the 2.5.9 I'm using now.

[Kevin]

Anyone?

My WebDev.WebServer.exe remains open after tests finish running and
every time I run them again it will fail.
Has anyone faced the same problem before?

[Jonas Bandi]

I can reproduce the problem. This is an issue.
We will look into it.
Can u use [BeforeFeature] in the meanwhile? This should get called...

While it definitely is an issue, that the hook is not called, I think
generally it is not such a good idea to rely on "After-Test"/TearDown
events (also in Unit- and Integration-Tests), since they only get
called if things are working as expected... I think there should be
some functionality in the "Before-Test"/SetUp events that guarantees
the proper environment. In your case isn't it possible to check for
the running WebServer.exe in the [BefortestRun] and do noting
respectively kill it there?

BTW. I would be interested in your usage of WebDev.WebServer.exe in
combination with SpecFlow and Acceptance Testing in general. Could you
sum up your setup and your experience (tips, deployment issues,
ui-automation ...). Maybe in a blog post that we could link on the
wiki (or just here on the lits).

--
mail: jonas...@gmail.com
web: www.jonasbandi.net
blog: blog.jonasbandi.net
twitter: twitter.com/jbandi

[Kevin]

ok, for now I will kill the process before starting the server, until
the issue gets fixed.


As for your question...

In my team we are using SpecFlow in a small project we started working
on.

We are using an IoC Container (Castle) to not only resolve components
for the application through configuration, but also to instantiate
acceptance test-specific components (the steps and other things),
depending on the type of test we're running.
So, for instance, if we execute the nant build script to run
Controller-level tests, certain properties are set so the Controller-
test steps implementations are registered in the container.

On the other hand, if we decide to test using WatiN, we run the nant
script to set properties and register components for browser specific
tests: WatiN browser, (which can either be IE or Firefox if we wish),
the server (we're using the local .Net WebDev.WebServer.exe, which we
wrap in a class that starts it using a System.Diagnostics.Process
object, and the browser steps implementations.


In terms of how the projects are structured, this is how we did it:

* Acceptance.Features.csproj

-- Global.cs (contains hooks such as BeforeTestRun, BeforeScenario
etc.)
The code within each hook simply calls the corresponding method in
the ITestFixtureSetup interface. The implementation of this interface
depends on whether we're running Controller level tests
(ControllerTestFixtureSetup.cs) or UI tests
(WatiNTestFixtureSetup.cs ..... here is where WatiN browser gets
instantiated and disposed and where the web server is started and
stopped).

-- One folder per each concept of the app. Eg: UserAccount, which will
contain all the .feature files for the UserAccount, such as
LogIn.feature, CreateAccount.feature, etc.

-- tests.properties file which has some general properties and the
type of ITestFixtureSetup implementation to use. This file is modified
at compile time by a nant script, to set the correct implementation of
the fixture setup to be used, among other acceptance test-specific
properties.


* Acceptance.Steps.csproj
-- in here we have one step file for each of the concerns concepts
above, and the methods are decorated with the [Given], [When], [Then]
attributes to bind the steps in the feature files.

-- These steps are just a wrapper to the concrete implementation in
the 2 projects below, either for Controller level test or UI.
Which implementation is used is determined at compile time by running
nant copies some files and sets some properties that tells the IoC
Container which implementation to use.

-- Each of the step files implement an interface
For instance, we have this for the user account steps:

[Binding]
public class UserAccountStepsWrapper :
StepsWrapperBase<IUserAccountSteps>, IUserAccountSteps
{
[Given(@"I am in the log in page")]
public void Given_I_Am_In_The_LogIn_Page()
{
Steps.Given_I_Am_In_The_LogIn_Page();
}

...etc.
}

The StepsWrapperBase is simply:

public abstract class StepsWrapperBase<T>
{
protected StepsWrapperBase()
{
Steps = IoCProvider.Resolve<T>();
}

protected T Steps { get; private set; }
}


* Acceptance.Steps.Controller.csproj
- each step file implements the interface defined in the project
above, and the steps will be called if the IoC container has
configured the Controller-level tests components.

* Acceptance.Steps.UI.csproj
- each step file implements the interface defined in the project
above, and the steps will be called if the IoC container has
configured the Browser-level tests components.
- The browser used is also registered in the Castle config file, so we
can also run tests for IE and Firefox if we wish, depending on a
property defined in the IoC framework.


Sorry if my explanation is a bit messy... if time permits and I can
put something more clear in a blog, I will.


Kevin

> mail: jonas.ba...@gmail.com

[Jonas Bandi]

I opened a bug for this:
https://github.com/techtalk/SpecFlow/issues/issue/26

--
mail: jonas...@gmail.com