Re: Struggling with TestCaseSource and TestCaseData when more than 1 method argument to pass in 'Arguments' override...

[Michael]

Woah, woah, woah: hold the phone. That's a lot of code: I got to the point where you're yield-returning an "adapter" pattern?

I'm not sure what the intent of the adapter pattern is, or how that necessarily plugs into NUnit, but in my experience, something like this is sufficient: perhaps your adapter pattern feeds the TestCaseData themselves?

private IEnumerable MyTestCases
{
  get
  {
    foreach (var testCase in GetMyTestCases())
        yield return testCase;
  }
}

private static IEnumerable<TestCaseData> GetMyTestCases()
{
  var testCases = new List<TestCaseData>();
  //TODO: Fill in the TestCaseData with values matching the types of your Test arguments.
  return testCases;
}

That's it: pretty straightforward. Of course, you need to name the test case in TestCaseSource, which I like to do in a private static class, public const string fashion (up to you at your discretion).

Don't know it helps you: you might also take a gander at Combinatorial: I've found this a very helpful decoration feeding test case values to tests without needing to worry about as much TestCaseData except in a handful of richer cases: like the Combinatorial is HUGE, or requires richer description than simpler primitive types would permit.

Anyone else care to elaborate further?

HTH

Best regards,

Michael Powell

On Tuesday, October 2, 2012 11:29:15 AM UTC-5, brad.hehe.et wrote:

I have a working set of unit tests which use a custom TestCaseData entity along with the TestCaseSource attribute to utilize it...

I'll start with the example which I currently have working...

 

Tests are decorated with:

        [NUnit.Framework.TestCaseSource(typeof(ServiceTestCases), "AllTestAccounts")]

Tests expect - in this case - a single argument...

        public void ExistsByIDTest
            (
                ITestAccountInfo testAccountInfo
            )

And the value I am returning for that source...

        public IEnumerable AllTestAccounts
        {
            get
            {
                // Return the AllAccounts set as a set of TestCaseData via an IEnumerable...
                foreach (var testAccountInfo in ServiceTestAccounts.GetAllTestAccounts())
                {
                    yield return new TestAccountTestCaseDataAdapter
                        (
                            testAccountInfo
                        );
                }
            }
        }
 

Which returns this TestCaseData-based entity...

    /// <see cref="http://www.nunit.org/index.php?p=testCaseSource&r=2.5.9"/>
    public class TestAccountTestCaseDataAdapter
        : NUnit.Framework.TestCaseData
    {
        /// <summary>
        /// 
        /// </summary>
        /// <param name="testAccountInfo"></param>
        public TestAccountTestCaseDataAdapter
            (
                ITestAccountInfo testAccountInfo
            )
        {
            // Validate the Arguments...
            if (testAccountInfo == null)
            {
                throw new ArgumentNullException("testAccountInfo");
            }
 
            // Reference the Arguments...
            _testAccountInfo = testAccountInfo;
        }
 
        private ITestAccountInfo _testAccountInfo;
        public ITestAccountInfo TestAccountInfo
        {
            get { return _testAccountInfo; }
            set { _testAccountInfo = value; }
        }
 
        /// <summary>
        /// The arguments to pass to the Test method.
        /// </summary>
        public new object[] Arguments
        {
            get { return new object[] { TestAccountInfo }; }
        }
 

So now I have a scenario where I want to pass more than 1 argument...  

Tests are now decorated/defined as such:

        [NUnit.Framework.TestCaseSource(typeof(FetchOperationTestCases), "AllTestCases")]
        public void FetchByIDTest
            (
                ITestAccountInfo testAccountInfo,
                FetchOperationTestCase testCase,

A new 'test cases' class was defines and exposes a simple property - just like my working example...

In this case each test case is another permutation on top of each Test Account permutation...

            public IEnumerable AllTestCases

            {

                 get

                 {

                     foreach (var testAccountInfo in ServiceTestAccounts.GetAllTestAccounts())

                     {                    

                         foreach (var testCase in FetchOperationTestCases)

                         {

                             var adapter = new FetchOperationTestCaseDataAdapter

                                 (

                                      testAccountInfo,                                

                                      testCase

                                  );

                             yield return adapter;

                         }

                      }

                  }

               }

 

As you can see this required a new 'TestCaseData' entity which includes the additional 'test case' information... inheriting/extending the existing custom one...

 

    public class FetchOperationTestCaseDataAdapter
        : TestAccountTestCaseDataAdapter
    {
        public FetchOperationTestCaseDataAdapter
            (
                ITestAccountInfo testAccountInfo,
                FetchOperationTestCase testCase
            )
            : base (testAccountInfo)
        {
            // Validate the inputs...
            if (testCase == null)
            {
                throw new ArgumentNullException("testCase");
            }
 
            // Reference the inputs...
            _testCase = testCase;
        }
 
        private readonly FetchOperationTestCase _testCase; 
        public FetchOperationTestCase TestCase
        {
            get { return _testCase; }
        }
 
        /// <summary>
        /// The arguments to pass to the Test method.
        /// </summary>
        public new object[] Arguments
        {
            get
            {
                return new object[]
                    {
                        TestAccountInfo,
                        TestCase
                    };
            }
        }
 

When I execute the above, I get the ReSharper/NUnit exception about Runtime Method binding from reflection... In this case it is the 'parameter count mismatch' exception I am getting...

[brad.hehe.et]

In addition to the above approach, since I had a working version when a single object is passed to the operation, I am attempting the following... I won't be thrilled to have to do this, but it's not a "big deal" to handle so it's a concession I'm willing to make...

 

 

        [NUnit.Framework.TestCaseSource(typeof(ContactServicesFetchOperationTestCases), "AllTestCases")]
        public void FetchContactByKeyTest
            (
                ContactServicesFetchOperationTestCaseArguments arguments
            )

Where the Arguments are simply a class to represent the 2 values...

    public class ContactServicesFetchOperationTestCaseArguments
    {
        public ITestAccountInfo TestAccountInfo { get; set; }
 
        public ContactServicesFetchOperationTestCase TestCase { get; set; }
 
    }

And the actual object[] to be passed to the operation is now built as follows...

        /// <summary>
        /// The arguments to pass to the Test method.
        /// </summary>
        public new object[] Arguments
        {
            get
            {
                return new object

[]
                    {
                       new ContactServicesFetchOperationTestCaseArguments()
                           {
                               TestAccountInfo = TestAccountInfo,
                               TestCase = TestCase
                           }
                    };
            }
        }

When I attempt this approach I get the same exception as above - which just amazes me how finnicky this is being:

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at NUnit.Core.Reflect.InvokeMethod(MethodInfo method, Object fixture, Object[] args)
at NUnit.Core.TestMethod.RunTestMethod(TestResult testResult)
at 
NUnit.Core.TestMethod.RunTestCase(TestResult testResult) 

Any good means of debugging this somehow?  I'm using ReSharper as my unit test runner if that matters or comes into the equation somehow....

[Michael]

I could be wrong, but I think I see the heart of your confusion: however you specify the arguments, whether "adapted" into the TestCaseData or however:

The arguments in the TestCaseData need to match the type and order of your test: generally I like to do this through the TestCaseData constructor.

If my test looks like this:

public void TestValues(int one, string two, bool three) {}

Then my arguments should look like this: new TestCaseData((int)1, (string)"two", (bool)true);

You don't specify the TestCaseData itself as a test case argument.

Yes?

[brad.hehe.et]

I'll look into the 'combinatorial' thing... That one is entirely new to me...

Otherwise, don't get too overly hung up on my naming there... the Adapter is just basically 'adapting' my other data sources (the TestAccountInfo and TestCase entities) into an NUnit TestCaseData inheriting object.  From my understanding, I need to build one of those TestCaseData-based objects that is responsible for describing the test case Name / Description and the 'arguments' that will be passed to the operation...  So in this case the IEnumerable is returning those TestCaseData-based objects...

 

For a little more info, I have two clases which act as the 'sources'... these are then deriving their contents from an external source (i.e. xml files) and building up some simple POCO entities...

 

    public static class ServiceTestAccounts
    {
        

        /// <summary>
	/// Returns the list of 'all' named Test Accounts
	/// </summary>
	/// <returns></returns>
	public static List<ITestAccountInfo> GetAllTestAccounts()
	{
            _allTestAccounts = TestAccountInfoFactory.LoadFromXml
                 (
                       ".\\ServiceTestAccounts.xml"
                 );
 
	    return _allTestAccounts;
        }  

 

And...

public class FetchOperationTestCases   

   {

        public IEnumerable AllTestCases
        {
            get

            {
                // Return the AllAccounts set as a set of TestCaseData via an IEnumerable...
                foreach (var testAccountInfo in ServiceTestAccounts

.GetAllTestAccounts())
                {
                    foreach (var testCase in FetchOperationTestCases)
                    {
                        var adapter = new ContactServicesFetchOperationTestCaseDataAdapter

                            (
                                testAccountInfo,
                                testCase
                            );
 
                        yield return adapter;
                    }
                }
            }
        }

 

 From the NUnit.Org docs for TestCaseSource I see this information...

 

Constructing Test Cases

In constructing tests, NUnit uses each item returned by the enumerator as follows:

1. If it is an object implementing NUnit.Framework.ITestCaseData, its properties are used to provide the test case. In NUnit 2.5, this is done using reflection to allow compatibility with earlier versions that did not implement ITestCaseData.

   The following public fields or properties are used:

   Arguments

       An object[] representing the arguments to the method

   Categories

       An IList of categories to be applied to the test case.

   Description

       Sets the description property of the test

   ExpectedException

       Specifies a the Type of an exception that should be thrown by this invocation

   ExpectedExceptionName

       Specifies a the FullName of an exception that should be thrown by this invocation

   Properties

       An IDictionary of properties to be applied to the test case. Note that the values provided must be compatible with PropertiesAttribute. In particular, use of custom types or enums will cause      problems.

   Result

       The expected result to be returned from the method, which must have a compatible return type.

   TestName

       Provides a name for the test. If not specified, a name is generated based on the method name and the arguments provided

   Ignored

       If true, the test case is ignored.

   IgnoreReason

       Specifies the reason for ignoring this test case. If set to a non-empty string, then the test is ignored.

 

If it is an object[], its members are used to provide the arguments for the method, as in this example, which returns arguments from a named static field.

 

And...

 

TestCaseData Class

Although any object implementing ITestCaseData may be used to provide extended test case information, NUnit provides the TestCaseData class for this purpose. The following example returns TestCaseData instances from a data source in a separately defined class.

 

[brad.hehe.et]

On Tuesday, October 2, 2012 12:47:03 PM UTC-4, Michael wrote:

Don't know it helps you: you might also take a gander at Combinatorial: I've found this a very helpful decoration feeding test case values to tests without needing to worry about as much TestCaseData except in a handful of richer cases: like the Combinatorial is HUGE, or requires richer description than simpler primitive types would permit.

 

Just took a gander at Combinatorial... that'd be swell if the values didn't have to be inlined in the code there... so I notice there is a ValueSource...  So I'm not sure if attempting to go that route really buys me anything more necessarily... I'll see if I can't hack that in somehow...  I just need something that works so I'm not too picky at the moment...

 

 

[brad.hehe.et]

On Tuesday, October 2, 2012 12:47:03 PM UTC-4, Michael wrote:

private static IEnumerable<TestCaseData> GetMyTestCases()
{
  var testCases = new List<TestCaseData>();
  //TODO: Fill in the TestCaseData with values matching the types of your Test arguments.
  return testCases;
}

 

My understanding is this..  if you use the TestCaseData(arg1, arg2, arg3, etc, etc) constructor ---   it's internals would be pasing the contents of the object[] that is internally storing those constructor argument values via the 'Arguments' property to the operation that is being bound via reflection.  Therefore by inheriting from TestCaseData and overriding that property - we are essentially doing the same thing....

 

[Charlie Poole]

Hi,

Your test needs to accept the arguments provided by the TestCaseData.
In the code you provided, the test doesn't take any arguments at all.

Perhaps that single hint will help you... but let's go a bit further.

Why do you need an "adapter" based on TestCaseData? Are you trying
to add some feature that NUnit doesn't have? If so, it won't work because
NUnit is only looking for what it knows about, so adding features requires
going much deeper into NUnit itself.

OTOH, if you are creating the "adapter" (seems a misnomer) in order
to pass application-specific info to the test, you're on a false track. That's
what the arguments to the method are for. Either provide multiple
arguments with the info you need or create an adapter of some kind that
holds all of them, and make the test method take that as its argument.

Note that there is nothing about TestCaseSource that requires you to
use TestCaseData. You can pass whatever arguments are needed
directly, as some examples show. If you use TestCaseData for some reason
then set the Arguments property to the array of arguments you use.

Ouch! I finally got the end of your note and I see you are running with
Resharper. Generally, this works the same as running tests under
NUnit itself, but if you are messing with how NUnit finds the test cases
in the TestCaseData, you could be out of luck. The info about
implementing your own data object that's in the NUnit docs describes
how *NUnit* interprets the provided info. I have no idea what will
work for Resharper. Load your tests in the NUnit gui and see what
shows up.

Charlie

On Tue, Oct 2, 2012 at 9:29 AM, brad.hehe.et <brad.h...@gmail.com> wrote:
> I have a working set of unit tests which use a custom TestCaseData entity
> along with the TestCaseSource attribute to utilize it...
> I'll start with the example which I currently have working...
>
>
> Tests are decorated with:
>
> [NUnit.Framework.TestCaseSource(typeof(ServiceTestCases),
> "AllTestAccounts")]
>
>
>
> Tests expect - in this case - a single argument...
>
> public void ExistsByIDTest
> (
> ITestAccountInfo testAccountInfo
> )
>
> And the value I am returning for that source...
>
>
>
> public IEnumerable AllTestAccounts
> {

> get
> {
> // Return the AllAccounts set as a set of TestCaseData via
> an IEnumerable...
> foreach (var testAccountInfo in
> ServiceTestAccounts.GetAllTestAccounts())
> {

> /// <summary>
> /// The arguments to pass to the Test method.
> /// </summary>
> public new object[] Arguments
> {

> get { return new object[] { TestAccountInfo }; }
> }
>
>
>
> So now I have a scenario where I want to pass more than 1 argument...
>
> Tests are now decorated/defined as such:
>
>
>
> [NUnit.Framework.TestCaseSource(typeof(FetchOperationTestCases),
> "AllTestCases")]
> public void FetchByIDTest
> (
> ITestAccountInfo testAccountInfo,
> FetchOperationTestCase testCase,
>
> A new 'test cases' class was defines and exposes a simple property - just
> like my working example...
> In this case each test case is another permutation on top of each Test
> Account permutation...
>
> public IEnumerable AllTestCases
> {
> get
> {

> foreach (var testAccountInfo in
> ServiceTestAccounts.GetAllTestAccounts())
> {
> foreach (var testCase in FetchOperationTestCases)
> {
> var adapter = new

> FetchOperationTestCaseDataAdapter

> (
> testAccountInfo,
> testCase
> );
> yield return adapter;
> }
> }
> }
> }
>
>
>

> As you can see this required a new 'TestCaseData' entity which includes the
> additional 'test case' information... inheriting/extending the existing
> custom one...
>
>
>
> public class FetchOperationTestCaseDataAdapter
> : TestAccountTestCaseDataAdapter
> {
> public FetchOperationTestCaseDataAdapter
> (
> ITestAccountInfo testAccountInfo,
> FetchOperationTestCase testCase
> )
> : base (testAccountInfo)
> {
> // Validate the inputs...
> if (testCase == null)
> {
> throw new ArgumentNullException("testCase");
> }
>
> // Reference the inputs...
> _testCase = testCase;
> }
>
> private readonly FetchOperationTestCase _testCase;
> public FetchOperationTestCase TestCase
> {
> get { return _testCase; }
> }
>

> /// <summary>
> /// The arguments to pass to the Test method.
> /// </summary>
> public new object[] Arguments
> {
> get
> {
> return new object[]
> {

> TestAccountInfo,
> TestCase
> };
> }
> }
>
>
> When I execute the above, I get the ReSharper/NUnit exception about Runtime
> Method binding from reflection... In this case it is the 'parameter count
> mismatch' exception I am getting...
>

> --
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/nunit-discuss/-/5Ek1JYgvJygJ.
> To post to this group, send email to nunit-...@googlegroups.com.
> To unsubscribe from this group, send email to
> nunit-discus...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/nunit-discuss?hl=en.

[Michael]

You're too close to it. Take a break create some distance from it.

You don't need to specify TestCaseData as a test parameter: that's the wrapper that packages the test parameter (Arguments) themselves.

Can anyone else help to set Brad straight?

[brad.hehe.et]

Also - I think for my situation - allowing NUnit to do the combinatorial figuring isn't necessarily going to work since some combinations aren't going to be valid... I'd prefer explicit construction of the permutations to be something easily done in whatever the solution proves to be...

 

With that said, I'd still prefer to build-up the TestCaseData entity and simply pass the 2 arguments to the operation...

 

Seems like this should be such a trivial task...

 

[Michael]

Thanks Charlie!

[Charlie Poole]

Hi Brad,

I second Michael on this :-)

You're making it complicated. Don't be afraid to just erase (well comment out)
what you have and start over. Write a test method that takes some list
of arguments you need. Make it work with TestCase if that's possible -
it might not be possible, since inline attributes can only take some kinds
of arguments. Now switch it to use a TestCaseSource like this:

object[] mySource = new object[] {
new object[] { arg_a_1, arg_b_1, ... },
new object[] { arg_a_2, arg_b_2, ... },
...
};

[TestCaseSource("mySource")]
public void MyTest(string arg1, double arg2, ...)
{
// test code
}

Next, consider whether you need to use TestCase data for some of it's
features - perhaps you want to expect an exception on some test.

If so, modify the above

TestCaseData[] mySource = new TestCaseData[] {
new TestCase( arg_a_1, arg_b_1, ... ),
new TestCase( arg_a_2, arg_b_2, ... },
...
};

The test method doesn't change.

So next, maybe you need to use a method for the source, because
the static arrays don't work for you...

IEnumerable<TestCaseData> mySource()
{
// Normally this would be in some sort of loop, but for the
// sake of illustration let's write...
yield return new TestCaseData(arg_a_1, arg_b_1, ... );
yield return new TestCaseData(arg_a_2, arg_b_2, ... );
...
}

Note that TestCaseData is still optional, you could just as easily write...

IEnumerable mySource()
{
// Normally this would be in some sort of loop, but for the
// sake of illustration let's write...
yield return new object[] { arg_a_1, arg_b_1, ... };
yield return new object[] { arg_a_2, arg_b_2, ... };
...
}

or even...

IEnumerable mySource()
{
// Normally this would be in some sort of loop, but for the
// sake of illustration let's write...
yield return new object[] { arg_a_1, arg_b_1, ... };
yield return new TestCaseData(arg_a_2, arg_b_2, ... );
...
}

The key points are
1. Make your test method take one or more parameters, which you provide
2. If it doesn't work under Resharper, try using NUnit to see where
the error lies

Charlie

> --
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/nunit-discuss/-/iEWJVhL72DwJ.

[Michael]

From the horse's mouth. Doesn't get any simpler than that.

[brad.hehe.et]

 

I'm using the TestCaseData approach since it offers me some tailoring to the test name and description as well as arguments passed to the operation...   I'm going to drop the term adapter since it seems to be confusing the hell out of people...

 

Otherwise the TestCaseData approach seems like using it is a 1-stop-shop approach to meeting my needs versus the other combination of approaches...

 

As well, I was 'accepting' the arguments from the TestCaseData.Arguments property...

 

        [NUnit.Framework.TestCaseSource(typeof(ContactServicesFetchOperationTestCases), ContactServicesFetchOperationTestCases.AllTestCasesSourceName)]
        public void FetchContactByKeyTest
            (
                ITestAccountInfo testAccountInfo,
                ContactServicesFetchOperationTestCase testCase
            )

 

Now - here's the REAL problem...  NUnit is behaving in an incredibly 'dumb' manner when it comes to dealing with exceptions during it's method discovery/invocation here... IMO...

 

Turns out I had a problem in my XML file parsing (the new part for these TestCases) and it was throwing a MEANINGFUL and INFORMATIVE exception spelling out what exactly was wrong (build output location of the file wasn't where I expected it)...

 

BUT....  The exception was NEVER dealt with nor bubbled up... rather - NUnit still attempted to invoke the test method/operation even though when retrieving the IEnumerable from the TestCaseSource threw an exception...   I would expect the test runner to catch & rethrow my exception rather than proceeding...  So instead it gives a totally misleading exception....

 

{FACE-PALM}....

 

And we can't blame ReSharper here I believe....  The stack trace shows it's all in NUnit's code base - right?  I'll try and use another runner later when I'm not pressed for time...

 

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)

at NUnit.Core.Reflect.InvokeMethod(MethodInfo method, Object fixture, Object[] args)

at NUnit.Core.TestMethod.RunTestMethod(TestResult testResult)

at NUnit.Core.TestMethod.RunTestCase(TestResult testResult)

 

If it proves to be a 'ReSharper thing' I'll submit a bug report to them... 

[brad.hehe.et]

I'm happy to report... Simply fixing the exception that was being thrown --- that did the trick!

 

I knew it was supposed to work the way I coded... The runner just "got in the way" of normal troubleshooting!

 

Thanks to those who attempted to help!  I did at least learn about Combinatorial along the way!

[Michael]

It's not perfect, and Charlie and folks do an incredible job putting together a pretty (IMO) robust test framework.

I will suggest, though, if you're frustrated with NUnit, there are other test frameworks you could use. We're establishing with you, however: you *are* making this more complicated than it needs to be.

I will reiterate: it sounds like you're confusing TestCaseData with the test arguments themselves. You can for instance specify ISomeInterface as one of your TestCaseData arguments. Then your test arguments can be: MyTest(ISomeInterface someIntf, ...).

Not to worry: I was confused about it at first as well. Once you get the hang of it, though: it's like second nature.

[brad.hehe.et]

I hear ya... I've used others and NUnit is actually my preferred framework...  Just in this case - swallowing exceptions didn't exactly do me any favors...

 

The TestCaseData object approach is very powerful and serves my needs perfectly here... Another happy customer served!

[Michael]

I will mention one other observation: you're mentioning swallowing exceptions, which leads me to believe the problem isn't in the test framework, the problem is in the arguments you're passing.

You can pass a factory anonymous functions to your test methods as well. That way any exceptions that do occur as a consequence of the interfaces you factory create occur in the context of the test and not the test data.

That might help you isolate the object(s) under test.

[brad.hehe.et]

I'm all for helping the dev team understand what I encounter so I'll try and explain a little more... 

 

Also worth metnion is that this is the 1st version I initially started off with and is the one that finally 'worked' once I resolved the file location issue...

 

Given this test method...

 

        [NUnit.Framework.TestCaseSource(typeof(FetchOperationTestCases), FetchOperationTestCases.AllTestCasesSourceName)]
        public void FetchContactByKeyTest
            (
                ITestAccountInfo testAccountInfo,
                FetchOperationTestCase testCase
            )

Point to this member on the given Type for the test case 'source'...

        public virtual IEnumerable AllTestCases
        {
            get

            {
                // Return the AllAccounts set as a set of TestCaseData via an IEnumerable...
                foreach (var testAccountInfo in ServiceTestAccounts

.GetAllTestAccounts())
                {
                    foreach (var testCase in FetchOperationTestCases)
                    {
                        var adapter = new FetchOperationTestCaseDataAdapter

                            (
                                testAccountInfo,
                                testCase
                            );
 
                        yield return adapter;
                    }
                }
            }
        }

 
Which in turn uses this lazy-loaded property when building up the permutations...
 
        protected List<FetchOperationTestCase> FetchOperationTestCases 
        {
            get
            {
                if (_fetchOperationTestCases == null)
                {
                    return _fetchOperationTestCases = LoadFetchOperationScenarios();
                }
                return _fetchOperationTestCases;
            }
        }
 
The 'loading' of the XML file... Which is where the exception was being thrown...  and ultimately 'swallowed' by the mechanisms involved in executing the tests...
 
 
        protected virtual List<ContactServicesFetchOperationTestCase> LoadFetchOperationScenarios()
        {
            try
            {
                // Ensure the XML file exists...
                if (!System.IO.File.Exists(ContactServicesTestCasesFilePath))
                {
                    throw new TestCaseSourceException
                        (
                            String.Format
                                (
                                    "Unable to locate the testing document [{0}]",
                                    ContactServicesTestCasesFilePath
                                )
                        );
                }
 
 
So in this case - here's what I would have expected..... Go ahead - call me crazy! You won't be the first or last!    ;-)
 
The 'runner' would have encountered that exception when attempting to access the 'source' property that returns an IEnumerable.
The 'runner' therefore wouldn't have even attempted to invoke the test operation via reflection... since it caught an exception while preparing the 'test data' from the stated source...
The 'runner' would then have thrown its own exception (hopefuly something typed that would convey it's area of functionality), nesting my exception that it caught as the InnerException...
Instead we see a call stack that is complaining about the method signature mis-match...   Which sent me on a goose-chase looking at why my Argument's property 
that returns the object[] was incorrect - though in reality it was just fine...
 
If I had seen my Exception that stated - hey! dummy! you're file isn't found!  - then I would have zero'd in on it right away...
 
Hopefully that helps illustrate what 'went wrong' in my case...  and what could be done to improve it for the next guy...
 
The embarrasing part is this - I think I identified this same behavior back when I was getting my 1st go at using TestCaseSource working for just the ITestAccountInfo based operations... 
I just forgot all about it... it's not exactly intuitive that the real 'root cause' has nothing to do with the call stack we're seeing here...
 
Again - thanks for the help here!
 
 

[Charlie Poole]

In the scenario you describe, the exception should be caught by NUnit
and recorded in the
parameter set (an internal class) that holds the test case data. (That
is, ParameterSet
in the NUnit internals matches TestCaseData in the user-visible portion)

The test method should be instantiated and visible in the Gui (why I
suggested using
NUnit for this type of development) as a non-runnable test. The
exception message
and stack trace should also be available. No actual test execution
should ever occur
when running NUnit, but from your stack trace it seems that R# goes ahead and
tries to run the test. If it's a non-runnable test, that's an error on
their part. OTOH,
if the exception is just thrown away and the test is marked as Runnable, then
it's an NUnit problem.

While you are not modifying NUnit internals, you are creating a
derived class that
must comply with the requirements of NUnit's internal processing. That's why I
suggested using aggregation rather than inheritance. What your doing is not
wrong - although additionally complicated - but you are going to be exposed
to errors that don't show up for the average user. When developing new
techniques like this, I strongly advise running some tests under NUnit itself
rather than under a 3rd party runner. If nothing else, it forces us to pay
more attention to your problems. :-) As far as having the time, it seems like
only a matter of minutes to fire up NUnit and load the tests.

From past experience with R# and its developers I know that their goal
is to make
anything that works in NUnit work with their runner. If you report a problem to
them, I imagine one of their first questions will be whether it works
with NUnit.

Charlie

> --
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/nunit-discuss/-/0CSlsto6yF0J.

[brad.hehe.et]

Just popped into the install dir and ran nunit-x86.exe to see what happens...

 

I started with 'breaking' the XML loading as it was orignially...

 

With the exception being thrown from my XML loading code... I get nothing - just tests didn't pass... I don't see anything related to the actual exception/call stack anywhere... I even enabled Verbose logging.  The path it says the log file will be located at doesn't even exist (on WIn 7)...

 

[ Google isn't letting me attach a 2nd picture now...  The tree-node for the test doesn't expand to show all test cases is what I wanted to show here... ]

 

 

Once I 'fix' the code so that the filename is valid again...  The nun-x86 runner now 'sees' all the test cases (ReSharper appears to 'probe' at execute time whereas NUnit's own runner appears to do so when building the list of tests - which I personally prefer this approach)....

 

 

 

I can't execute the actual tests but I suspect that is due to some other unrelated aspect... 

[brad.hehe.et]

Here's the scren shot I couldn't attach on prior message...  Weird...

 

[Charlie Poole]

Hi Brad,

Comments inline...

On Tue, Oct 2, 2012 at 12:50 PM, brad.hehe.et <brad.h...@gmail.com> wrote:
>
> Just popped into the install dir and ran nunit-x86.exe to see what happens...
>
> I started with 'breaking' the XML loading as it was orignially...
>
> With the exception being thrown from my XML loading code... I get nothing - just tests didn't pass... I don't see anything related to the actual exception/call stack anywhere... I even enabled Verbose logging. The path it says the log file will be located at doesn't even exist (on WIn 7)...

Looking at your second note, I see FetchContactByKeyTest nested under
FetchContactByKeyTest. The outer one represents the method. The inner
one is a test case with no arguments shown because we didn't get any.
If you right-click that test and click properties, you should see why.
Also, the test should be red in the tree before you even run it
(assuming NUnit 2.6 or 2.6.1) and should stay red when you try to run
it. I don't see that in your display so I guess you're running an
older version.

Regarding the log file, there's a bug indicating that it won't create
the directory if it doesn't already exist. It will be fixed in 2.6.2
but if you create the directory yourself you should see the logs
there.

> [ Google isn't letting me attach a 2nd picture now... The tree-node for the test doesn't expand to show all test cases is what I wanted to show here... ]
>
>
> Once I 'fix' the code so that the filename is valid again... The nun-x86 runner now 'sees' all the test cases (ReSharper appears to 'probe' at execute time whereas NUnit's own runner appears to do so when building the list of tests - which I personally prefer this approach)....
>
>
>
>

> I can't execute the actual tests but I suspect that is due to some other unrelated aspect...

Your tests are marked as if they were ignored. Looking at the
properties will probably tell you more about it.

Charlie

> --
> You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/nunit-discuss/-/reCrbWlS7rEJ.