Testing the Gallio solution.

[Yann Trevin]

Hello Gentlemen,

I have a few questions about building and testing the Gallio solution.

The "Test.bat" located in the root of the solution works fine, but it
runs all the tests of the solution; so it requires a long time to
complete. What I want to do is to run only a few given tests (let's
say those in the MbUnit.Tests project for instance). So I have tried
to use TestDriven.NET, but I get an exception thrown during the setup
of the testing environment, by the Windsor IoC kernel about an
missing/unspecified instance name for IReportManager.

Did I miss some required configuration file to drop in the bin folder?
Or is it related to a conflict between the tested solution and the
actual version of Gallio (build 285) installed in my program files
directory (and which is obviously invoked by TestDriven.NET)

In fact, I'm a bit confused about the required environment I need to
test Gallio (You know, testing a testing framework!) Some enlightment
would be very much appreciated.

Thanks a lot in advance.
Yann.

[Jeff Brown]

Testing a testing framework is a indeed a little touchy at times.

One of the trickier bits is that there can only be one version of Gallio
running at a time. So that means the version used to the run tests must be
the same as the one being tested.

At the same time, there are some tricky bits associated with ensuring that
all of the plugin components can be located when running a debug version
because the source file layout on disk is different from the eventual
deployed format in C:\Program Files. So the plugin resolution strategy is a
little different for DEBUG vs. RELEASE builds.

When working in Visual Studio, here's how to get things done.

1. Do a full solution build.

You may notice that some components have dependencies on third party
programs such as ReSharper. The build system tries to detect whether these
programs are installed and will cause parts of the build to fail with a
warning if not. Don't worry, these components are generally optional.

2. How to run tests using TestDriven.Net.

You need to ensure that TestDriven.Net is configured to load the Gallio
TDNet add-in. To do so, run the
src\Extensions\TDNet\Install-TDNetRunner.bat file. Unfortunately if you are
running 64-bit Windows, you may need to configure the registry by hand
instead.

When this is done, you should be able to run tests in Visual Studio using
TD.Net.

3. How to run tests using R#.

Running tests with R# is a little trickier because of how R# loads its
add-ins. Generally you'll only want to do this when testing the R# runner
because you'll need to restart Visual Studio each time you recompile.
Instructions on how to do this are in the
src\Extensions\ReSharper\Gallio.ReSharperRunner folder.

4. Using Gallio.Echo.

The most convenient way to run large numbers of tests in DEBUG mode outside
Visual Studio is to use Gallio.Echo.

Use the src\Extensions\Echo\Run-Echo.bat file to run Gallio.Echo from the
source tree. The batch file adds some command-line options to ensure that
Echo can resolve its plugins.

5. Using Gallio.Icarus.

You can also use Gallio.Icarus to run tests. Unfortunately this only works
if Icarus was built in DEBUG mode or if you run it from the "build" folder
generated by a RELEASE build. The reason is again due to how plugins are
resolved but unlike Echo we don't have an option in Icarus yet that allows
us to override the plugin paths. For that reason we usually have to depend
on the special strategy used in DEBUG builds unless we shuffle the files
around.

To use Icarus in DEBUG mode, run a full build in Visual Studio. Then launch
src\Extensions\Icarus\bin\Gallio.Icarus.exe.

Hope this helps.
Jeff.

[Yann Trevin]

Thank you very much, Jeff. It helps a lot.

[Graham Hay]

For the record, it is possible to add plugin directories for the GUI (you get the Gallio install folder for free). Under Tools->Options there is a "Plugin directories" section (see attached). Any folders added to this list are stored in an Icarus.settings file under App_Data and will be re-used until removed (you need to re-start the app before any changes take effect).

2008/7/18 Jeff Brown <jeff....@gmail.com>:

[Jeff Brown]

Nifty.

Some of this stuff may change when we move to Mono.Addins (or similar) because it can maintain its own plugin registry file on disk with all of the paths.

 

Jeff.

---

From: galli...@googlegroups.com [mailto:galli...@googlegroups.com] On Behalf Of Graham Hay
Sent: Wednesday, July 23, 2008 5:44 AM
To: galli...@googlegroups.com
Subject: Re: Testing the Gallio solution.