Help Need how to run the nunit tests parallel

[Madhu Palagani]

HI,

am using Nunit.3.0 rc, to run tests parallel

while running the tests using  [Parallelizable(ParallelScope.Children)] at test method, test are running one by one

please suggest me how to run the tests parallel one each another, below is my code 

namespace NunitTestProject

{

    [TestFixture]

    [Parallelizable(ParallelScope.Fixtures)]

    public class TestSet001

    {

        [Test]

        [TestCase("2")]

        [TestCase("3")]

        [TestCase("4")]

        [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test001(string dataRow)

        {

          SeleniumTest.RunSeleniumTestInLocalChrome(dataRow);

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test002()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test003()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test004()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test005()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test006()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test007()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test008()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test009()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test010()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test011()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

        [Test]

     [Parallelizable(ParallelScope.Children)]

        public void TestSet001Test012()

        {

            SeleniumTest.RunSeleniumTestInLocalChrome();

        }

}

[Rob Prouse]

Madhu,

I am pretty sure that running test methods in parallel is planned but not complete yet. AFAIK, only TestFixtures are currently run in parallel. That information is in the dev notes, but needs to be added to the documentation.

Charlie Poole, can you confirm this?

Rob

--
You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nunit-discus...@googlegroups.com.
To post to this group, send email to nunit-...@googlegroups.com.
Visit this group at http://groups.google.com/group/nunit-discuss.
For more options, visit https://groups.google.com/d/optout.

--

Rob Prouse

 

I welcome VSRE emails. Learn more at http://vsre.info/

[Charlie Poole]

Rob is correct. We had actually hoped to implement this for the 3.0
release, but it's more complicated than you might think and we had to
push it back to the next release after 3.0.

You are able to run different fixtures in parallel right now. You
should be careful that they don't share any resources before doing so.
ParallelScope.Children will basically only work down to the fixture
level. Setting the scope on methods has no effect at all.

I'll create an issue for the docs to make this clear.

[Jeremy Dunn]

I've been following the 3.0 release closely, and I guess I was overly-optimistic that parallel execution at the test level would make it into 3.0(sounds like the dev team was as well). We are currently using MbUnit to distribute a large number of selenium tests across a number cloud machines in parallel, but we have a strong urge to get back onto NUnit. Do you currently have a ticket open to track parallel execution? I'm not seeing anything in the backlog. How high is it on your project priority list? Broadly speaking, is this something we could expect to see in 2016?

[Charlie Poole]

The feature is tracked here: https://github.com/nunit/nunit/issues/164


As you can see, it's tracked under milestone "future" which is not a
good sign for you. :-(


However, although there are some issues under milestone 3.2, we have
not really spent time to triage all those issues and make up our minds
about what it should include. We'll be doing that immediately after
the final release of NUnit 3.0.

Here's the thing... any open source project prioritizes according to
the particular itches that it's contributors want to scratch. Or at
least according to those things they can imagine needing. Our
conclusion, up to now, has been that most users don't really need to
run test cases in parallel. In fact, we have concluded that parallel
test cases might be kind of dangerous for the average user. So
although it's on the list, it's not currently a high priority.

If that assumption turned out to be untrue for Selenium users, it's
kind of up to you folks who use it to explain it to us. I've never
used Selenium and I don't do web development with .NET.
Coincidentally, many of the heavy contributors to NUnit are backend or
desktop developers. We have a tendency to favor our own "flavor" of
development.

There are two things you can do to overcome that tendency:

1. Explain things to us realy well. Is there some reason you cannot
structure your tests so that you use fixtures rather than methods as
the "thing" that needs to run in parallel? I have noticed that several
folks who have asked about this feature are using Selenium. So, is it
intrinsic to Selenium or is it merely the way you are accustomed to
using it. That would be really useful info for us and we would use it
in our prioritization.

2. Take it a step further and become one of us. It would be great to
have one or more strong contributors who do this type of development.

Charlie

[Jeremy Dunn]

I believe it boils down to the fact that I'm using the TestCaseSource attribute to feed multiple sets of dynamic(decided at run time) test mutators(my word) to a [Test] method that fundamentally change what the test is doing in each instance, or rather how and where it is running. An (overly)simplified example:

class CreateTests : IEnumerable

    {

        public IEnumerator GetEnumerator()
        {
            foreach(*some logic determinable at run time*)
                {
                    *set mutators according to said logic*
                    yield return new object[] { browser, version, platform, site, language };
                }
        }

[Test, TestCaseSource(typeof(CreateTests)), Parallelizable]
public void LoadSomePage(string browser, string version, string platform, string site, string language)
...

Now that I've talked through this, it occurs to me that I could move the test factory logic into thread safe queue(s) and convert my [Test]s to [TestFixture]s that pull from those, but it seems like quite a roundabout way to accomplish parallelization, if this solution would work. Is there an easier way that I'm missing to pass dynamic parameters into a TestFixture similar to TestCaseSource for Tests? The simplest explanation I have to why I was looking at [Test]s rather than [TestFixture]s to run in parallel is that it requires the smallest amount of rework to move from our currrent implementation of MbUnit.

I'll admit I did find myself browsing the project earlier to see how hard it may be to extend it myself.. :)

[Charlie Poole]

OK. That makes sense: the problem is at least partly that TestFixtures
have fewer facilities for being data-driven than test cases.

Although we haven't talked about it much, NUnit 3.0 now has a
TestFixtureSourceAttribute, which is used to generate arguments for
creating test fixtures. Take a look at that.

It seems to me that the ideal solution for testing across multiple
browsers might be to have a data-driven test fixture with a parameter
that chooses the browser and a series of contained tests to exercise
it.

[Jeremy Dunn]

Thanks for the heads up Charlie, this really looks like exactly what I was looking for! I've been attempting to prove that this is possible today, and I think I'm very close, but I'm having trouble accessing the data from TestFixtureSourceAttribute; I can get parameterized TestFixtures to work without using a factory object to feed the data:

        [TestFixture("one")]

        [TestFixture("two")]

        [TestFixture("three")]

        public class TempTestFixure

        {

            private string myString;

            public TempTestFixure(string MyString)

            {

                this.myString = MyString;

            }

            [Test]

            public void MyTest()

            {

                Debug.Write(myString);

            }

        }

However, I'm unable to get it to work with a factory(Dynamic or Static) feeding TestFixtureSource():

        public class TestFactory : IEnumerable

        {

            public IEnumerator GetEnumerator()

            {

                for (int x = 0; x < 3; x++)

                    yield return new object[] { x.ToString()};

            }

        }

        static object[] TestFactory2 = 

        {

            new object[] { "one" },

            new object[] { "two" },

            new object[] { "three" } 

        };

        [TestFixture, TestFixtureSource("TestFactory"/*2*/)]

        public class TempTestFixure

        {

            private string myString;

            public TempTestFixure(string MyString)

            {

                this.myString = MyString;

            }

            [Test]

            public void MyTest()

            {

                Debug.Write(myString);

            }

        }

It always complains of "OneTimeSetUp: No suitable constructor was found". I've tried providing it with such a method since it seems so insistent:

            [OneTimeSetUp]

            public void doNothing()

            {

            }

But that results in the same error. I then also tried both of these trying to appease it:

            [OneTimeSetUp]

            public void doNothing(string testData)

            {

            }

            [OneTimeSetUp]

            public void doNothing(object[] testdata)//This one wouldn't even make sense since I just want one yielded result, but I digress...

            {

            }

But those just throw "OneTimeSetUp: Invalid signature for SetUp or TearDown method: doNothing" It seems I'm just missing something small, but I can't seem to nail it down... Unfortunately I'm having trouble finding any documentation on this attribute. Can you see what I may be missing??

Thanks,

Jeremy

[Charlie Poole]

Hi Jeremy,

The "OneTimeSetUp:" message prefix doesn't refer to a method with
[OneTimeSetUp] simply means that the error occured during NUnit's one
time setup of the test, which includes construction of the fixture.
The rest of the message, "No Suitable constructor was found" means
that no constructor for the fixture class was found to match the data
provided. IOW, it's telling you your TestFixtureSource is not working.
As you already knew. :-)\

Your code looks to me as if it should work. It's very much like our
code for testing the feature - see TestFixtureSourceData.cs in the
testdata folder. I'll give it a try when I get a chance.

In researching this, I discovered that we never created a
documentation page for this attribute! It's on my todo list now.

Charlie

PS: We don't use the term Factory and we use Static and Dynamic in a
different way - we don't yet have what we call dynamic. No harm in
your use of mbunit terms, but you should be clear that the semantics
may not be the same.

[Charlie Poole]

Here is the documentation for TestFixtureSourceAttribute:
https://github.com/nunit/nunit/wiki/TestFixtureSource-Attribute

Turns out it was on the wiki but not in the index of attributes.

charlie

[Jeremy Dunn]

Thanks for throwing the documentation up Charlie! I was able to get this working earlier in the week after some hair pulling... I wound up in a cycle of causing more problems that I was solving when trying to get TestFixtureSource() to work, so eventually I gave up and started from scratch, at which point everything started working. I honestly don't know what the original issue was; along the way I did discover that nuget package manager was rebuilding nunit 2.6 in addition to 3.0 despite me uninstalling so I suspect that was part of the problem. Long story short- the code from my previous comment does work as expected :). Thanks again for the patient assistance!!!

[Jeremy Dunn]

I did just notice that TestFixtureSource isn't inherited, this makes my code a little messy when reusing a my source.. can you elaborate on why source attributes aren't inherited? Would I be hurting myself by allowing inheritance on them?

On Wednesday, November 11, 2015 at 10:18:21 PM UTC-6, charlie wrote:

[Charlie Poole]

Jeremy,

Since "inheritance" means several different things when dealing with
attributes, could you clarify what you would like to do?

Charlie

[Jeremy Dunn]

Basically I was expecting to be able to do this:

public class SomeTestTF : UITest
{
   [Test]
    public void Sometest
   {
       //do test
   }
}

[TestFixtureSource(typeof(UITestsSource)), Parallelizable]
public class UITest
{

}

 

After working through this though, I've discovered that I didn't really want to do it in the first place... So it's safe to just ignore this :D.

[Sandeep Kumar]

NUnit has now support for Parallel Test Execution for tests in different TestFixtures.  For a step-by-step introduction - Refer this [Video Playlist](https://www.youtube.com/playlist?list=PLQ9cQ3JqeqU8yWhANCUDH-OiotwQ4VaZI)

[Charlie Poole]

The posted video is a very basic introduction to testing web pages
using Selenium and NUnit. There is a little about
ParallelizableAttribute at the end. It seems to be oriented toward
testers, since the web pages already exist before the testing begins.
Within those limitations it's a nice intro.

@Sandeep, I suggest you label it for what it is. Believe it or not,
there are still some non-web folks around! Also, please post anything
like this on it's own thread rather than tagging it on to some other
discussion.

Charlie