Selenium grid - run test on all registered nodes

[Marko Vucinic]

I am looking of way to run 1 test on multiple nodes. For example, I have multiple VMs with different IE versions. I want one test to run on all of them instead of bloating my code with cloned versions of the same test in order to achieve: one test -> multiple versions run(in parallel) scenario

Is that achievable, can I get that creating custom capability matcher maybe?

Thank you

Marko

[Krishnan Mahadevan]

Marko

Test execution is NOT an attribute of the Grid. Grid only distributes sessions across all nodes. 

If you need sessions to be created on all nodes you need to have your test generate that many create session requests and the only way of doing it is by ensuring number of tests being executed concurrently is equal to the total number of sessions that can be handled by all nodes put together. 

So you need to do this in your test code and not at the grid. 

What I don't understand is, why do you need to run a test on all nodes ?

Assuming that your nodes differ from each other is OS Flavor, Browser and browser version combinations it makes sense. But you should still only be doing those combinations at your test side. What difference doesn't it if a test runs on NodeA or NodeB when both of them are running IE 9 on windows 7 for eg. 

~ Krishnan

iSent. iPhone. iReget.iTypos!

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/12661dc8-0327-4048-aa09-ccc461091a79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marko Vucinic]

Purpose of running one test on multiple nodes is checking if one functionality working on all versions of IE for example. So if I have multiple versions of IE, each on one node, then if I send test to hub with matching capability ie, it should match all of my nodes.

This way I am testing that one functionality is working on all versions of some browser and if not, I can tell on which one it doesn't, and...it works in parallel so I'm cutting testing time which is really important for automation testing...

I understand from documentation and posts, that test execution is not attribute of the grid, but distribution to nodes is, and here is only matter of design choice, are you stopping with first matched node or you passing test to all nodes that complies with criteria(for example, type of browser)...

Anyway, back to the subject, problem here is that in order to run one test on multiple versions of the browser I have only two options:

-run same test in loop with different parameters(different versions of same browser) - which will run sequentially, so if I have 5 nodes, only one at the time is running test, other are idle

-clone same test and pass different version parameter in each clone, then run using test engine which supports parallelism - in this case it will run in parallel, but my test code now is pretty messy

Both solutions are compromise and kind of a hack because grid doesn't support fully parallelism in its core, I was wondering is there a way to solve this...

Thank you

[Krishnan Mahadevan]

Marko

I dont think you are understanding what is the intent of the Grid.

The intent of the Grid is NOT to facilitate parallel test executions or serial test executions. That is the attribute of a Test Runner and NOT of the grid.

What you are essentially asking for is: Grid when it gets a new session request, should basically trigger duplicate new session requests (based on the capabilities that were requested in the original new session request), which to me sounds like a whacky behavior if it is associated with the Grid.

What you should be doing is, power your test via a Data provider, wherein you are providing the set of capabilities as parameters via a data source and your test runner takes care of iterating the combinations. 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/bdd95e6c-fe03-42e4-b0e4-d2576b98eb69%40googlegroups.com.

[Marko Vucinic]

Hey Krishnan, I understand what is intent of the grid, I just don't understand why is like that by design. Grid is not part of some other system and brought here as a support to distributed testing, it is built from it is core to support selenium distributed testing. In architecture where you have nodes as separates computers, I don't understand design decision: one test per node, especially in scenario of automation testing which are by their nature slow, so running time is really important. And if you look at how are people solving this issue and make grid to work in parallel, they are using either solution that you have mention which is not supported by all test engines(run this test with this parameters multiple times and run it in parallel) or they using some kind of whacky foreach loops in order to iterate tests.

[Krishnan Mahadevan]

Marko,

>>>>>I don't understand design decision: one test per node, especially in scenario of automation testing which are by their nature slow, so running time is really important.

There is NO SUCH design decision being made. No one is stopping you from having your nodes support more than 1 test session running concurrently. But what the Grid WILL NOT do is, run the same test again and again in parallel. 

The test code asked for just 1 test session. So why do you think the Grid should be creating duplicate sessions which may or may not differ from each other in capabilities.

I dont think the Grid is at fault for a given technology not having the inherent ability of supporting data driven or parallel tests. I believe it is a problem with the technology that you are playing with.

All said and done, I believe I have tried to articulate the justifications behind why the grid is currently as it is, and why it should not be as how you are asking it to be.

If my answers aren't convincing enough, I am more than glad to step aside and let someone else from the core selenium dev group [ yes I am still NOT part of that elite clan ] to give you an answer which perhaps answers your question in a satisafactory manner !

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/683904e3-1508-4a27-a617-476e78f2520f%40googlegroups.com.

[Shawn McCarthy]

Marco, the way I handle running a test against different browsers (in parallel) is by running my test suites multiple times (using Jenkins). I can run 3 Jenkins jobs at the same time, which run the same test suite, but with different browsers (IE8, IE9, IE10 for example). I pass in the browser name as a JVM argument. So each time the test gets pointed at the Selenium Grid Hub, it has a different browser capability, and gets directed at the different nodes.

Hope this helps !

[Marko Vucinic]

I guess that you can specify in Jenkins, run this test this number of times with this parameters, but run it in parallel, I heard python testing framework can do the same. I am using MSTest and that is not possible unfortunately. I kind of disagree that there is no design decision being made but I guess we took this too deep, I have started from different point of view, expecting certain kind of behavior and grid is just not build that way, I have kind of solution which includes running tests in parallel with reversed order of parametered foreach, so I am getting parallelism, bah, I was hoping that I can skip that kind of awkwardness...

Thank you

Marko 

[Felicia Hsieh]

TestNG might help. You can use the invocationCount attribute (threadPoolSize must be used in conjunction).

@Test(invocationCount = 5, threadPoolSize = 5)

public void myTest() {

System.out.println("Thread #"+Thread.currentThread().getId()+" myTest("+(getCount()+1)+")");