AutoTest MSTest runner is not instantiating classes in the correct order

[Craig Shea]

First off, I want to say that up until now, I have really enjoyed using Mighty Moose. Don't get me wrong, I don't dislike using Mighty Moose at this point, I'm just a bit frustrated that it's not working correctly. But, as with all software, there are bound to be bugs; and it looks like I was the unfortunate soul to find one. Anyway, keep up the good work, I really like the tool.

So, let me explain about the "bug" that I found.

I've been playing around with the idea of having test "contexts", implemented as nested test classes, in order to keep the setup methods small and focused on only containing the data needed for a set of tests. I'm writing some code to implement a Category class. The invariants of the contract class are that a category name cannot be null or whitespace, a category parent cannot be null, and a subcategory of the category cannot be added if the subcategory is a null object or the Null Object Pattern Category.None. Here's a partial listing of the test class that demonstrates the issue:

[TestClass]

public class CategoryTests

{

    private string PARENT;

    private string CHILD;

    [TestInitialize]

    public void InitializeCategoryTestsContext()

    {

        PARENT = "Parent";

        CHILD = "Child";

    }

    [TestMethod]

    [ExpectedException(typeof(InvalidCategoryNameException))]

    public void Cannot_create_a_Category_with_a_null_string_for_a_name()

    {

         // I'm trying out different stuff--a factory may indeed not be appropriate; but my code architecture is not at issue here, though the execution of the tests are...so bear with me. :)

        CategoryFactory.CreateCategory(null);  

    }

    [TestClass]

    public class ParentCategoryContext : CategoryTests

    {

        private Category parent;

        [TestInitialize]

        public void InitializeParentCategoryContext()

        {

            parent = CategoryFactory.CreateCategory(PARENT);

        }

        [TestMethod]

        public void A_top_level_category_returns_its_name_when_ToString_is_called()

        {

            Assert.AreEqual(PARENT, parent.ToString()); // This passes in VS when using MSTest test runner, but when run using AutoTest.Net, PARENT is null, causing the test to fail an invariant.

        }

    }

}

Now, since the field PARENT is never used in the outer test class, yes, I could have put the field in the inner class (actually, I should do that, regardless, but when I first wrote the test, I planned on using the field in the outer class. Not only that, if the classes were instantiated correctly, the field should have been inherited by the inner class anyway, making this whole diatribe extraneous to begin with :) ). When I do put the PARENT field in the inner class, the tests in that class pass as expected:

[TestClass]

public class CategoryTests

{

    private string CHILD;

    [TestInitialize]

    public void InitializeCategoryTestsContext()

    {

        CHILD = "Child";

    }

    [TestMethod]

    [ExpectedException(typeof(InvalidCategoryNameException))]

    public void Cannot_create_a_Category_with_a_null_string_for_a_name()

    {

         // I'm trying out different stuff--a factory may indeed not be appropriate; but my code architecture is not at issue here, though the execution of the tests are...so bear with me. :)

        CategoryFactory.CreateCategory(null);  

    }

    [TestClass]

    public class ParentCategoryContext : CategoryTests

    {

        private string PARENT;

        private Category parent;

        [TestInitialize]

        public void InitializeParentCategoryContext()

        {

            PARENT = "Parent";

            parent = CategoryFactory.CreateCategory(PARENT);

        }

        [TestMethod]

        public void A_top_level_category_returns_its_name_when_ToString_is_called()

        {

            Assert.AreEqual(PARENT, parent.ToString()); // This passes in VS and in AutoTest.Net when PARENT is declared and defined in this class.

        }

    }

}

It seems like AutoTest is just finding all classes marked with the TestClassAttribute without taking into consideration it's scope (e.g. that it may be nested in another class) and properly instantiating it based on its scope.

While in this instance, I was able to "fix" the failing tests so that they pass, this was a very easy scenario--the field in question was not used in the outer class. Had it been a situation where the field was used in both the outer and inner class, the inner tests would fail (or, I'd have to declare and define the same fields in the inner class, which is exactly what I'm trying to avoid; or, a third option, declare new classes for each "context", repeating the required setup code from other contexts where needed in order to ensure all tests pass--which one might argue could be cleaner, despite the code duplication--no "deep" nesting and no "hidden" setups). In any event, invariably, some will choose the nesting style I have chosen (at least, for now).

Anyway, as I said in my intro, keep up the good work. I really like the tool. Thanks again!

[Craig Shea]

Ok, so I'm back, because now this really is biting me :)

So, if I have a parent category, I also have a child. And given the invariants I mentioned for my Category class above, when testing parent/child relationships, I need both a parent and a child. So, I created a nested ChildCategoryContext class that inherits from its outer parent class ParentCategoryContext so that I can use both the parent category (created in the outer class) and the child category (created in the inner class).

Because of the above mentioned bug, I am unable to "share" (read, inherit) the outer class's implementation because its constructor is never called as part of the scope chain of ChildCategoryContext. :(

I'm hoping you guys can identify this soon and get it fixed. Guess I should check out your contributors policy on GitHub and see if I can't spot the problem :)

[Craig Shea]

And I'm back here one last time.

I figured out how to make my tests pass using nested classes. There's still a bug in AutoTest.Net, but at least I'm able to "work around" it in a way that I find satisfactory. After running the code shown below and having my tests pass, I believe that AutoTest.Net is properly instantiating classes. What it's not doing, is executing the inherited class hierarchy's methods marked with the TestInitializeAttribute attribute.

The code below is partly from the code I posted above that both MSTest and AutoTest.Net pass, but includes the further nested class ChildCategoryContext. The lines highlighted in bold are what makes the Child Category tests pass:

[TestClass]

public class CategoryTests

{

    private string CHILD;

    [TestInitialize]

    public void InitializeCategoryTestsContext()

    {

        CHILD = "Child";

    }

    [TestMethod]

    [ExpectedException(typeof(InvalidCategoryNameException))]

    public void Cannot_create_a_Category_with_a_null_string_for_a_name()

    {

         // I'm trying out different stuff--a factory may indeed not be appropriate; but my code architecture is not at issue here, though the execution of the tests are...so bear with me. :)

        CategoryFactory.CreateCategory(null);  

    }

    [TestClass]

    public class ParentCategoryContext : CategoryTests

    {

        private string PARENT;

        private Category parent;

        [TestInitialize]

        public void InitializeParentCategoryContext()

        {

            PARENT = "Parent";

            parent = CategoryFactory.CreateCategory(PARENT);

        }

        [TestMethod]

        public void A_top_level_category_returns_its_name_when_ToString_is_called()

        {

            Assert.AreEqual(PARENT, parent.ToString());

        }

        // More tests follow...

        [TestClass]

        public class ChildCategoryContext : ParentCategoryContext

        {

            private string CHILD;

            Category child;

            [TestInitialize]

            public void InitializeChildCategoryContext()

            {

                base.InitializeParentCategoryContext(); // This was the most important line, besides moving child category-specific fields into this class

                CHILD = "Child";

                child = CategoryFactory.CreateCategory(CHILD, parent); // parent is inherited from outer class, it works only when calling base.InitializeParentCategoryContext(). Perhaps nested classes are properly instantiated, but the test initializer functions are not being run.

            }

            [TestMethod]

            public void A_subcategory_returns_its_parents_name_with_a_path_separator_and_its_own_name_when_ToString_is_called()

            {

                Assert.AreEqual(string.Concat(PARENT, "/", CHILD), child.ToString());

            }

        }

    }

}

Hope that helps you guys out.

[Svein Arne Ackenhausen]

Not to worry, I'll be the first to admit there is bugs in there :) Just to make sure I get this right. MSTest can access non static private fields in the parent class from it's nested classes? Or am I misunderstanding something here? AFAIK that shouldn't even compile?

-Svein Arne

--
Du mottar denne meldingen fordi du abonnerer på Google-gruppen «AutoTest.Net».
For å melde seg av denne gruppen og slutte å motta e-poster fra den, send en e-post til autotestnet...@googlegroups.com.
Du finner flere alternativer på https://groups.google.com/groups/opt_out.

--
Best regards
Svein Arne Ackenhausen

Blog - http://ackenpacken.blogspot.com/
Twitter - http://twitter.com/ackenpacken