Parallel execution

212 views
Skip to first unread message

Kuzhali Velmurugan

unread,
Jun 12, 2018, 7:52:54 AM6/12/18
to Selenium Users
Hi,

I am trying to execute the test classes in parallel in Chrome instance.
I had set thread count as 3, three instances of browser got opened.
But the username and password gets kkeyed in in the same instance.

Below is the BaseSet up , where I initialize the driver:

public static WebDriver driver;

@Parameters({ "browserType" })
@BeforeClass(alwaysRun = true)
public void initializeTestBaseSetup(@Optional("chrome") String browserType) throws MalformedURLException {
    setDriver(browserType);
}

@AfterClass(alwaysRun = true)
public void baseSetupLogOut() {
    driver.quit();
}

public WebDriver getDriver() {
    return driver;
}

private void setDriver(String browserType) throws MalformedURLException {
    switch (browserType) {
        case "chrome":
            driver = initChromeDriver();
            break;
        case "firefox":
            driver = initFirefoxDriver();
            break;
        case "IE":
            driver = initIEDriver();
            break;
        default:
            driver = initChromeDriver();
    }
}

private WebDriver initChromeDriver() throws MalformedURLException {
    if ("remote".equalsIgnoreCase(mode)) {
        DesiredCapabilities capability = DesiredCapabilities.chrome();
        driver = new RemoteWebDriver(new URL(hubUrl), capability);
        driver.manage().window().maximize();
    } else if ("local".equalsIgnoreCase(mode)) {
        System.setProperty("webdriver.chrome.driver", "drivers/Chrome/chromedriver.exe");
        ChromeOptions chromeOptions = new ChromeOptions();
        chromeOptions.addArguments("--start-maximized");
        driver = new ChromeDriver(chromeOptions);
    } else {
        throw new IllegalArgumentException("Unsupported mode '" + mode + "'");
    }
    driver.get(miningUrl);

    return driver;
}

And I try to do login action, before class (this is in different class and test classes extend this class):
@BeforeClass(alwaysRun = true)
public void baseSetupLogin() {
    commonUtils.login();
    objDemoHomePage = new DemoHomePage(driver);
    objLogin = new LoginPage(driver);
}


Kindly help me to resolve the problem and execute tests in parallel.

Thanks,
Kuzhali

Cassian Raja Thomas

unread,
Jun 12, 2018, 8:37:41 AM6/12/18
to seleniu...@googlegroups.com
Try keeping all the implementations of @BeforeClass in BaseTest class itself and have all your tests extend it

Thanks & Regards,
Cassian Raja .T

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/c523b49d-dbb1-41b5-90ad-d71264732c1e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Scott Babcock

unread,
Jun 12, 2018, 8:54:56 PM6/12/18
to Selenium Users
You're storing your driver references in a `static` field, which means that there's only ever one. Consequently, the references to the first two drivers are being overwritten and lost. You could try storing your driver references in a ThreadLocal, which will provide the per-method storage you want for parallel execution. This is how I handled a similar requirement in one of my frameworks. The code can be found here: ExecutionFlowController.java

This capability is used by my Selenium framework to pass driver references and other test-specific values form phase to phase: TestNgBase.java

Chaitanya Deshpande

unread,
Jun 12, 2018, 10:38:44 PM6/12/18
to seleniu...@googlegroups.com
Please share your TestNG XML or code.

Thanks.

Regards,
Chaitanya Deshpande
+91 9028332930

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

Scott Babcock

unread,
Jun 12, 2018, 11:30:59 PM6/12/18
to Selenium Users
Another challenge you'll probably run into with parallel execution in TestNG is that thread count isn't necessarily going limit the number of session that will be created to that exact count. It's quite common to see a run that specifies a thread count of 10 to end up launching 12 or more tests simultaneously, with a corresponding number of browser sessions. To keep from overtaxing your system resources, you end up dialing the thread count way down, which limits your throughput.

In my Selenium framework, I acquire drivers from a local instance of Selenium Grid. The Grid hub configuration provides explicit control over the maximum number of session that will be active at any given moment. This driver management strategy has several other benefits as well. All you need to do to switch to a different driver or add a new driver is to update your configuration. You never need to crack open your code.

If you're interested, the project is found here: [Selenium Foundation](https://github.com/Nordstrom/Selenium-Foundation)

On Tuesday, June 12, 2018 at 4:52:54 AM UTC-7, Kuzhali Velmurugan wrote:

Krishnan Mahadevan

unread,
Jun 12, 2018, 11:49:11 PM6/12/18
to seleniu...@googlegroups.com

Scott,

 

>>>> It's quite common to see a run that specifies a thread count of 10 to end up launching 12 or more tests simultaneously, with a corresponding number of browser sessions.

That to me would be a straight forward bug either in the Test code that is responsible for browser instantiation or the underlying TestRunner. It should be fixed. For me that wouldn’t be a reason for trying to check for alternatives.

 

>>>> In my Selenium framework, I acquire drivers from a local instance of Selenium Grid.

Yes, I remember going down that rabbit hole myself a couple of years back. For local executions (which is mostly the case when you are doing test development and would like to see the browser kick off and watch your test in action to debug), requiring a local instance of Selenium Grid (the one that you spin off via your code, wherein you fire up a hub, fire up a node, wire them together) to come up before browser instances can be allocated to me sounds like too much of an ask.

 

Local executions should be seamless. They should be making use of the more specific RemoteWebDriver variants such as ChromeDriver, FirefoxDriver etc., instead of having to rely on RemoteWebDriver. From a framework perspective it makes sense to try and keep everything standard to always a RemoteWebDriver. In that case, one should try using the DriverService variant (but it’s a lot of overhead and one is better off with using ChromeDriver directly for e.g.,) Something like the below sample.

 

public static void main(String[] args) throws IOException {
  DriverService service = ChromeDriverService.createDefaultService();
  service.start();
  URL url = service.getUrl();
  RemoteWebDriver driver = new RemoteWebDriver(url, new ChromeOptions());
  driver.get("http://www.google.com");
  System.err.println("Title " + driver.getTitle());
  driver.quit();
  service.stop();
}

 

 

>>>> The Grid hub configuration provides explicit control over the maximum number of session that will be active at any given moment.

If this is the only reason why you are using a local grid, then you could consider alternatives. One such alternative would be to guard your browser instantiation and cleanup via methods that are guarded by Semaphores for e.g., Am sure there are other elegant ways of achieving this as well.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

Scott Babcock

unread,
Jun 13, 2018, 3:13:51 AM6/13/18
to Selenium Users
Krishnan,

Different browsers expend different amounts of system resources. If you're targeting multiple browsers with your automation, this requires driver-specific management of the maximum instances of any given browser. Why re-invent this, when Grid provides this as a standard feature? I don't know what exact effect the thread count value has on the inner workings of TestNG. This isn't something I need to understand. If I logged a bug with the TestNG team that stated, "I get 12 tests running simultaneously when I set the thread count to 10," they're likely to tell me that this is normal behavior. Why is this significant? My concern is to limit the maximum number of browsers open at any given time, which is exactly what Grid provides.

Rabbit hole? The framework fires up a local Grid in a matter of seconds. Performance has never been an issue. The ability to change the targeted driver and add new browsers by simply updating your configuration is huge. The elimination of behavioral anomalies between local and remote operation avoids a lot of frustration.

Regards,
= Scott =

Krishnan Mahadevan

unread,
Jun 13, 2018, 3:25:51 AM6/13/18
to seleniu...@googlegroups.com

Scott,

 

>>>> If I logged a bug with the TestNG team that stated, "I get 12 tests running simultaneously when I set the thread count to 10," they're likely to tell me that this is normal behavior. 

 

I am one of the committers in TestNG. Trust me, I am not going to come back to you and say that if you set a thread count of 10 and if you notice 12 tests running simultaneously that’s normal behavior. That is a bug. A straight forward bug. Thread-count of 10 in TestNG would mean that TestNG would at any given point in time only run 10 @Test annotated methods at a time. I say this because I know how TestNG works. So, if you have an example wherein you are seeing things otherwise, then it needs to be investigated and fixed. So please help share a sample for that.

 

>>>> Different browsers expend different amounts of system resources. If you're targeting multiple browsers with your automation, this requires driver-specific management of the maximum instances of any given browser.

 

Yes. Absolutely correct. But that doesn’t mean one would need to rely on a remote execution mechanism (which is what the Grid is actually meant for) to basically do that management when it can very well be managed by a sophisticated test runner (such as TestNG for example).

 

>>>> The ability to change the targeted driver and add new browsers by simply updating your configuration is huge.

A well-designed framework should be able to manage this by still letting you specify the browser flavor via a configuration and also specify if it’s a local or a remote run and yet achieve them by toggling between FirefoxDriver/ChromeDriver/InternetExplorerDriver etc., for local runs and using RemoteWebDriver only for remote runs. Its not that difficult.

 

>>>> The elimination of behavioral anomalies between local and remote operation avoids a lot of frustration.

 

I must correct you here. There are no behavioral anomalies between local and remote operation. The grid is just a sophisticated load distribution mechanism. Nothing more and nothing less.

If you take a closer look at the Selenium node implementation you will notice that it still resorts to invoking the same FirefoxDriver or ChromeDriver but via reflection. Just because a test runs a UI automation testcase locally by employing a selenium grid spawned via code, doesn’t get rid of any frustrations that managing a Grid is associated with.

Because things such as ephemeral port exhaustion, selenium node leaking memory due it running for a longer time are still going to come up in a remote execution environment which you will never see in a locally spun off grid via code and whose lifetime is short.

Scott Babcock

unread,
Jun 13, 2018, 5:29:37 AM6/13/18
to Selenium Users
Krishnan,

If you specify thread count and data provider thread count together, you get wildly varying numbers of tests running at any given moment. I don't think this is a bug. This is just multi-threading in action. The objective of running tests in parallel is to maximize throughput. The implementations of the tests themselves don't require significant resources. It's the browsers that really devour resources. The goal is to run as many session as possible without causing the system to bog down. This concern is not the responsibility of the generic test framework. This should be handled by the scenario-specific implementation that runs on top.

Yes, Selenium Grid was originally designed for remote operation. As far as I know, this is true for every component of WebDriver. Why does the typical application of Grid preclude it from being used for local operations? You're advocating for the re-creation the functionality of Grid. What are the advantages of this approach that justify the time and effort this would require?

⇜Krishnan Mahadevan⇝

unread,
Jun 13, 2018, 6:34:29 AM6/13/18
to seleniu...@googlegroups.com
Not every component of webdriver was designed for remote execution. 

Selenium grid was the only one designed for remote execution and as far as I know ( from the RC days this is true ). 

There's a lot to the selenium grid than what you are suggesting. So it would be an understatement if one were to say that controlling the number of sessions in a different way would be equal to duplicating the selenium grid functionality. 

The idea is simple. Local executions are only used when a test is being created or built. For that one doesn't need anything more than a simple webdriver instantiation. 

All the benefits that you mentioned come into effect when one is basically running in a CI mode on when there's a build system involved which constantly runs tests. For that one wouldn't be using a programmatically spun off grid but would be routing their tests to an actually running grid in a remote environment. 

The idea is to use the tool for which it was meant for. 

A test and a data driven test are attributes of a test runner. Browser instantiation is completely separate from that. 

Building a simple logic that throttles instantiation for local executions ( this by itself is not required in my opinion because my premise is local execution is only for Dev mode and not for the actual test mode where tests are run in bulk) doesn't require any effort than some basic programming abilities ( which is anyways needed when one takes up the task of building a framework). 




For more options, visit https://groups.google.com/d/optout.
--

Scott Babcock

unread,
Jun 13, 2018, 1:05:13 PM6/13/18
to Selenium Users
public class ChromeDriver extends RemoteWebDriver public class EdgeDriver extends RemoteWebDriver public class FirefoxDriver extends RemoteWebDriver public class InternetExplorerDriver extends RemoteWebDriver public class OperaDriver extends RemoteWebDriver public class SafariDriver extends RemoteWebDriver

Seems to me that it's all designed for remote operation. Not only are you proposing that folks should re-implement existing functionality, you're also suggesting that "developer" code should differ from "production" code. 

Aside from that, the OP's question regards driver management when running tests in parallel. I posted my observation regarding the 'static' modifier on his `driver` field and its affect on the behavior of his code. I then informed him that the `threadCount` setting isn't guaranteed to provided one-to-one control over the number of tests that are executing at any given moment when running in parallel. I pointed him at the solution I've used to achieve explicit control over the maximum number of browser sessions that will be active at any given moment. If the OP finds this useful, great. If not, perhaps someone else with a similar problem will. Regardless, I've provided accurate information and proven solutions.

⇜Krishnan Mahadevan⇝

unread,
Jun 13, 2018, 1:11:21 PM6/13/18
to seleniu...@googlegroups.com
The remote operation for all the drivers is the server component (chromed river/gecko driver etc). Remote webdriver lays down the basics that are common across all webdriver implementation and the customisations take care of starting and stopping the server. I believe that is perhaps the more apt way of looking at the implementations. 

Sure, no second thoughts on if you answered the OP question. Not debating that aspect at all. 

Am only putting forth my point of view based on my experiences with building http://selion.io 

All said and done, I guess we can wrap up our discussion with "let's agree to disagree". 


For more options, visit https://groups.google.com/d/optout.

Kuzhali Velmurugan

unread,
Jun 14, 2018, 5:28:50 AM6/14/18
to Selenium Users
Hi,

Removing the static reference to driver solved the issue.

Thank you Scott and Krishnan.

Kuzhali Velmurugan

unread,
Jun 14, 2018, 8:10:58 AM6/14/18
to Selenium Users
Hi,

I am facing another issue. 

Null Pointer Exception is being thrown in screenshot capture method. Is it because the driver object is not initialized properly?

public class ScreenshotOnFailure extends BaseSetup implements IHookable {

@Override
public void run(IHookCallBack callBack, ITestResult testResult) {

    callBack.runTestMethod(testResult);
    if (testResult.getThrowable() != null) {
        try {
            takeScreenShot(testResult.getMethod().getMethodName());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

@Attachment(value = "Failure in method {0}", type = "image/png")
private byte[] takeScreenShot(String methodName) throws IOException {
    return ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);
}


The driver is being used from the base class.

Getting exception at the highlighted line.
I have used it as a listener to take screenshot on failure.

Kindly share your thoughts.


Thanks,
Poonkuzhali

⇜Krishnan Mahadevan⇝

unread,
Jun 14, 2018, 11:07:53 AM6/14/18
to seleniu...@googlegroups.com
Do you see the problem when you use TestNG 6.14.3 (latest released version of TestNG)

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7fd57389-4729-425d-86fd-f3e0f11d03ed%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Kuzhali Velmurugan

unread,
Jun 14, 2018, 12:41:27 PM6/14/18
to seleniu...@googlegroups.com
Yes, I face the issue while using TestNG, have added the listener in the XML file. 

But am not using the latest version of TestNG.

You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/cEBYO-RaYis/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.

To post to this group, send email to seleniu...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--

Regards,
V.Poonkuzhali

⇜Krishnan Mahadevan⇝

unread,
Jun 14, 2018, 12:45:05 PM6/14/18
to seleniu...@googlegroups.com
Please retry using 6.14.3 and kindly post back your findings. 

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANOO4NbXRsd14pC9opKfwwfNdmm-CBE0bN136-cqNv_EeVi4nw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Kuzhali Velmurugan

unread,
Jun 14, 2018, 12:51:40 PM6/14/18
to seleniu...@googlegroups.com
Ok, Thank you. I will try with that version and post on Tuesday(we have long weekend). 


For more options, visit https://groups.google.com/d/optout.
--

Regards,
V.Poonkuzhali

Kuzhali Velmurugan

unread,
Jun 22, 2018, 8:40:22 AM6/22/18
to Selenium Users
Hi,

It didnt seem to work with that version of testNG (6.14.3).


On Thursday, 14 June 2018 22:21:40 UTC+5:30, Kuzhali Velmurugan wrote:
Ok, Thank you. I will try with that version and post on Tuesday(we have long weekend). 
On Thu, 14 Jun 2018 at 10:15 PM, ⇜Krishnan Mahadevan⇝ <krishnan.mahadevan1978@gmail.com> wrote:
Please retry using 6.14.3 and kindly post back your findings. 

On Thu 14 Jun, 2018, 22:11 Kuzhali Velmurugan, <kuzhali.v...@gmail.com> wrote:
Yes, I face the issue while using TestNG, have added the listener in the XML file. 

But am not using the latest version of TestNG.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/cEBYO-RaYis/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--

Regards,
V.Poonkuzhali

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANOO4NbXRsd14pC9opKfwwfNdmm-CBE0bN136-cqNv_EeVi4nw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.
--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/cEBYO-RaYis/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
--

Regards,
V.Poonkuzhali

Krishnan Mahadevan

unread,
Jun 22, 2018, 11:27:30 PM6/22/18
to seleniu...@googlegroups.com

Please share a full fledged simple standalone test, that can be used to reproduce the problem.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

From: <seleniu...@googlegroups.com> on behalf of Kuzhali Velmurugan <kuzhali.v...@gmail.com>
Reply-To: <seleniu...@googlegroups.com>
Date: Friday, June 22, 2018 at 6:10 PM
To: Selenium Users <seleniu...@googlegroups.com>
Subject: Re: [selenium-users] Re: Parallel execution

 

Hi,

 

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.


To post to this group, send email to seleniu...@googlegroups.com.

--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/cEBYO-RaYis/unsubscribe.

To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.


To post to this group, send email to seleniu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.


For more options, visit https://groups.google.com/d/optout.

--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/cEBYO-RaYis/unsubscribe.

To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

--

 

Regards,

V.Poonkuzhali

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/ae194197-eb1d-49e9-9d9f-792480af08ff%40googlegroups.com.

Kuzhali Velmurugan

unread,
Jun 26, 2018, 4:25:00 AM6/26/18
to Selenium Users
Hi,

Actually, the web driver instance in screenshot capture listener class was null.
Initially, as the web driver instance was static, I didn't face this issue.
Now I did a workaround, added an after method in base setup to capture the screenshot when test fails.
This resolved the issue.

Kuzhali Velmurugan

unread,
Jul 5, 2018, 6:26:58 AM7/5/18
to Selenium Users

Hi ,



By adding the below 2 highlighted lines, resolved the issue when the screen capture method is added as a listener.

And web driver instance will not be null and screenshot gets attached to allure report.


Reply all
Reply to author
Forward
0 new messages