Waiting for a slot in the Grid queue while running many tests.

[OxygeneIV]

Hi,

I'm launching lots of tests and want the Hub to just relay me to approriate node for each test when a slot is free.

Since I also involve AutoIt etc on the nodes I have limited the nodes to accept only one session at a time which then of

course will produce a large Hub queue,

As tests may take a long time to finish and the number of tests are many, the last tests registering to the hub will be executed (in theory)

several hours later, i.e. it'll be at the bottom of the list.

This then, forces me to set the CommandTimeout to a too high value I think (e.g. 10 hours). Otherwise I get a timeout error while waiting for a free node

from the RemoteDriver contructor.

I do not want the  general CommandTimeout to be more that 120 secs or so.

Should not the driver request timeout be separate from the general CommandTimeout used later when a session has been granted?

Using a shorter CommandTimeout or setting shorter "newSessionWaitTimeout" , I could "retry" to fetch a session in the tests using a loop, 

but then my test would loose its place in the Hub-queue for each failure.

Anyone with an idea how to solve this dilemma. 

Env. : Selenium 2.35, .NET C#

 

/M

[Luke Inman-Semerau]

Is your test suite the only one running against your grid? If so, then I might suggest setting your thread concurrency (number of tests run at the same time) to the number of nodes that are in your hub. 

If not, I don't see anything too wrong (other than wasted resources waiting) for setting a higher timeout value. Although reducing the concurrent tests running will reduce that timeout as you'll likely only need to wait the time it takes for a test to complete if your concurrent thread number is close to the number of available nodes. (There are programatic ways to query the grid for how many nodes are available)

-Luke

--
You received this message because you are subscribed to the Google Groups "webdriver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webdriver+...@googlegroups.com.
To post to this group, send email to webd...@googlegroups.com.
Visit this group at http://groups.google.com/group/webdriver.
For more options, visit https://groups.google.com/groups/opt_out.

[Krishnan Mahadevan]

Adding to what Luke elaborated, 

You might want to look at checking if this is absolutely necessary 

>>>>Since I also involve AutoIt etc on the nodes I have limited the nodes to accept only one session at a time which then of course will produce a large Hub queue,

If it is, then you would need to start finding more nodes for your Hub and match your parallel thread count to the number of nodes in your Hub.

Thanks and Regards

Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

[OxygeneIV]

Ok, thanks.

If I was launching the tests using threads from a single suite it might be more easily solved.

But I've created a testrunner program wrapping NUnit accepting a regexp of filtered  tests to be launched for a desired capability/environment

E.g. (simplified)

runnerManager.exe  -firefox -mac -tests "login.*"

runnerManager.exe  -firefox -win -tests "login.*"

runnerManager.exe  -chrome -win -tests "login.*"

runnerManager.exe  -firefox -win -tests  "^test_mail"

...and 50+ more

 
All of these are launched "at the same time" to be able to use all nodes effectively so I end up with separate testprograms ( runnerManager.exe ), each  requesting a node to work with.

If one would look at the task manager it would show lots of runnerManager.exe-processes , most of them just waiting for a free node.

Actually, it is also so that for e.g. suite 1 above, "-firefox -mac -tests 'login.*'", which may be 20 testcases, each  has a setup/teardown , where setup is requesting a node and teardown quits the driver...

All in all, one could say that I forward each and every testcase to the Hub in one go, and let the Hub handle them.