Discovering all tests from a dll using Nunit API
Rahul
I have a specific requirement where I need to discover all tests from
a test dll. Each test method is decorated with [Test] attribute, also
we have parameterized tests. I need to discover tests manually rather
than just loading the dll in nunit so that I can distribute these
tests to get executed over multiple test machines.
One way to do it would be to write my own reflection code that cracks
open the test assembly and figures out all test methods. However this
would mean that I duplicate the code that is already in nunit.
So my question is: Does nunit expose public API that I can use to
simply discover all test cases from a given test dll?
I tried following approaches but both return zero results:
// Approach1
TestSuiteBuilder suiteBuilder = new TestSuiteBuilder();
Test suite = suiteBuilder.Build(new TestPackage(testDll));
// Approach2
TestAssemblyBuilder assemblyBuilder = new
TestAssemblyBuilder();
TestSuite suite = assemblyBuilder.Build(testDll, true);
Is there a way to do this?
Thanks,
Rahul.
Charlie Poole
The answers are somewhat different for NUnit 2.x and NUnit 3.0. I'll cover
both of them. NUnit 2.x is stable but also at the end of its life after 2.6.
NUnit 3.0 is still changing and it's at the start of it's life. If you
want to make
internal changes to NUnit 3.0, we might keep them.
On Wed, Jul 6, 2011 at 5:46 PM, Rahul <rahulr...@gmail.com> wrote:
> Hello,
> I have a specific requirement where I need to discover all tests from
> a test dll. Each test method is decorated with [Test] attribute, also
> we have parameterized tests. I need to discover tests manually rather
> than just loading the dll in nunit so that I can distribute these
> tests to get executed over multiple test machines.
I'll assume this is arbitrary distribution for load purposes and that all
tests are independent of one another.
> One way to do it would be to write my own reflection code that cracks
> open the test assembly and figures out all test methods. However this
> would mean that I duplicate the code that is already in nunit.
Yes, and that you would have to update when NUnit changed.
> So my question is: Does nunit expose public API that I can use to
> simply discover all test cases from a given test dll?
NUnit 2.x: no public/published api. There are public methods you
can use, but they are internal and subject to change. That said, we
try not to break you.
NUnit 3.0: There is a TestEngine with a published API. It will support
distributed execution but doesn't do so yet. There is an initial implementation
of filtering by test name but it needs some work.
> I tried following approaches but both return zero results:
> // Approach1
> TestSuiteBuilder suiteBuilder = new TestSuiteBuilder();
> Test suite = suiteBuilder.Build(new TestPackage(testDll));
> // Approach2
> TestAssemblyBuilder assemblyBuilder = new
> TestAssemblyBuilder();
> TestSuite suite = assemblyBuilder.Build(testDll, true);
>
> Is there a way to do this?
The above is NUnit 2.5 and will remain essentially the same in 3.0.
Both are low-level classes. That is, they operate after NUnit has already
decided whether to load and run tests in a separate AppDomain or Process.
If you use them for discovery only, you should probably run your code
in a separate AppDomain. You could also use NUnit's TestDomain to
do the same thing - it would create an appdomain and then load the
tests for you. TestDomain and the other higher-level classes in NUnit
are in the nunit.util assembly.
Either approach (or TestDomain) requires that you initialize the services
used by NUnit. Take a look at the startup code for the console runner
to see an example of initializing services. Without that, you won't
find any tests at all.
The same classes exist - possibly with name changes - in NUnit 3.0
but they are now buried in the framework itself and only accessible
from inside the framework. To use NUnit 3.0 you could either
access the framework directly or use the test engine. The test engine
is the best long-term option, since the framework is only designed
to be used by a driver within the engine. Nevertheless, there is a
test program (direct-runner) that demonstrates how to access it
directly.
None of the above addresses how you will actually execute the tests.
My suggestion would be to have each of your server machines load
the entire assembly but to control what is executed by passing filters
as each new execution started.
Hope this helps.
Charlie
> Thanks,
> Rahul.
>
> --
> You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
> To post to this group, send email to nunit-...@googlegroups.com.
> To unsubscribe from this group, send email to nunit-discus...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nunit-discuss?hl=en.
>
>
Dinis Cruz
the NUnit APIs, that can be 'easily' reached and invoked using
reflection.
But Rahul, if all you want is where are the tests, that can easily
retrieved from reflection
Dinis
On Jul 7, 6:29 am, Charlie Poole <nunit...@gmail.com> wrote:
> Hi Rahul,
>
> The answers are somewhat different for NUnit 2.x and NUnit 3.0. I'll cover
> both of them. NUnit 2.x is stable but also at the end of its life after 2.6.
> NUnit 3.0 is still changing and it's at the start of it's life. If you
> want to make
> internal changes to NUnit 3.0, we might keep them.
>
Rahul
Thanks a lot for your reply. I was able to accomplish what I wanted to
do using your tip. I first initialize all Nunit services and then
following code fetches all test cases from a dll:
TestDomain domain = new TestDomain();
TestPackage package = new TestPackage("Multiple Assemblies
Test");
package.Assemblies.Add(testDll);
domain.Load(package);
ITest loadedSuite = domain.Test;
TestNode test = loadedSuite.Tests[0] as TestNode;
Will you be keeping these classes (TestDomain, TestNode), properties
and methods public in Nunit 3.0. I want to confirm if this code will
work when we upgrade to Nunit 3.0.
The reason I ask is we will be using this piece of code in our real
production system. If this breaks we will be blocked.
Thanks,
Rahul.
On Jul 6, 10:29 pm, Charlie Poole <nunit...@gmail.com> wrote:
> Hi Rahul,
>
> The answers are somewhat different for NUnit 2.x and NUnit 3.0. I'll cover
> both of them. NUnit 2.x is stable but also at the end of its life after 2.6.
> NUnit 3.0 is still changing and it's at the start of it's life. If you
> want to make
> internal changes to NUnit 3.0, we might keep them.
>
Charlie Poole
NUnit 3.0 is a complete redesign and rewrite of NUnit, so you would not
be doing the same thing at all to execute tests. However, executing tests
will be much simpler because we have an Api to support it.
Here's some examples based on the current state of the code, which
you can view or download at http://launchpad.net/nunit-3.0.
// Get an interface to the test engine
ITestEngine engine = TestEngineActivator.CreateInstance();
...
// Run tests in a package
TestEngineResult result = engine.Run(package, listener, filter);
...
// Get a test runner and use it to load tests and do multiple runs
ITestRunner runner = engine.GetRunner();
TestEngineResult loadResult = runner.Load(package);
if (!loadResult.IsError)
{
TestEngineResult result1 = runner.Run(listener, filter1);
TestEngineResult result2 = runner.Run(listener, filter2);
}
Filtering is not yet implemented, but I plan to unify selection by name,
category, properties, etc. into a single API, using an XML representation
of the filter. Linq queries are also a possibility.
Charlie
Justin Toth
> > To unsubscribe from this group, send email to nunit-discuss+unsubscribe@googlegroups.com.
Charlie Poole
The result of Rahul's code is a single test node at the base of the
tree of tests. To find all test cases, you would need to descend the
tree recursively, capturing each test case you find.
Charlie
>> > > athttp://groups.google.com/group/nunit-discuss?hl=en.
>
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> https://groups.google.com/d/msg/nunit-discuss/-/nCanUK8fXDcJ.
> To post to this group, send email to nunit-...@googlegroups.com.
> To unsubscribe from this group, send email to