Monitor and gracefully kill or reuse zombie GeckoDriver instances

[Eugene S]

Hi all,

I have a question regarding the management of geckodriver zombie processes.

Selenium is designed to kill Geckodriver process when disposing of the webdriver instance like this: driver.quit(). 

In my testing framework, I have configured to call driver.quit() after each test.

In some cases however, the test terminates abnormally due to various reasons and then the finalizing steps (like quitting the driver) are not being executed. As a result I end up with multiple geckodriver processes that I can only kill manually.

Is there any good practice how to work around this problem? Perhaps monitoring the processes somehow and killing the ones that do not transfer any data?

Or maybe creating few instances and then reusing them?

I am running my tests on Grid so manipulating the process through command line won't work as it will only affect the grid hub.

There is an option of using taskkill to terminate all the geckodriver processes on a machine but this is not a good solution since:

1. I am running my tests through Selenium Grid so this will only affect the grid hub and not the nodes.
2. There are multiple tests that run simultaneously and if I use taskkill I will kill the ones that are still in use as well.
3. I don't think that killing the processes using taskkill is a generally a good practice I would like to find a more elegant solution. E.g. (1)geckodriver processes are monitored on the node and only terminated locally when they are not in use or (2)somehow reused.

Many thanks,

Eugene S

[⇜Krishnan Mahadevan⇝]

Eugene,

AFAIK, if your cleanup viz., driver.quit() is part of a cleanup activity as defined by your test runner (for e.g., part of an @AfterClass annotated method in Java, where @AfterClass is an annotation exposed by the TestRunner TestNG), then the cleanup should happen on its own.

The Grid also has some cleanup logic that is exposed via parameters called timeouts.

You can read more about them here : https://github.com/SeleniumHQ/selenium/wiki/Grid2#configuring-timeouts-version-221-required

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/796f6672-6b63-4a51-b3d6-12d08e91ee54%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eugene S]

Hi Krishnan,

Thanks a lot for your input.

In my case, I am describing a situation when the run was interrupted and I no longer have access to the remotely started geckodriver instance.

For example, in case when a critical error occurs during the creation of the remote webdriver instance so that the browser has started (and subsequently the geckodriver) but then the session is broken. In such scenario, executing `quit()` on driver instance will not work as the connection already lost.

Yes, I am aware of the grid timeouts but the question is how do I utilize them to help me detect "stale" geckodriver sessions.

Thanks,

Eugene 

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

[⇜Krishnan Mahadevan⇝]

Eugene,

You dont need to do anything extra apart from setting some sane values for those timeout parameters. The rest, the Grid infrastructure will take care.

So whenever there's no activity on a browser window for X Units of time due to either the client crashing (your test case is the client) or due to the browser going into a hung state (perhaps due to a rogue javascript) or due to your test case, opening a browser window and not doing anything on it, the cleanup of the browser (and thus the server component as well) will happen from the server side (The Selenium node in this case).

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/64a37de0-1b1d-4dc5-a71b-8e78bbbceead%40googlegroups.com.

[Eugene Shragovich]

Thanks again Krishnan but please note that I am talking about the geckodriver.exe process, not the actual browser window itself.

The browser closes fine, not problem. My issue are the remaining geckodriver processes that are now not bound to anything and keep adding up with each new browser window open.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/64a37de0-1b1d-4dc5-a71b-8e78bbbceead%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/5cQPSnkg__g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZLkzydLW0O6iGE0rXX%2B3vqJ4Q4O_gUEDGkWtt7Q175%2BF0g%40mail.gmail.com.

[⇜Krishnan Mahadevan⇝]

Eugene,

I understand that. One of the many things that happen as a result of a browser cleanup via driver.quit() is also to terminate the server process (geckodriver/chromedriver/IEDriverServer)

See DriverCommandExecutor.execute() implementation here  [this is where driver.quit() gets translated into http://localhost:port/shutdown on the server component to signal killing the browser process by the server process] which in turn calls DriverService.stop() to cleanup the process associated for a particular session.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Thu, Jan 18, 2018 at 2:04 PM, Eugene Shragovich <shrag...@gmail.com> wrote:

Thanks again Krishnan but please note that I am talking about the geckodriver.exe process, not the actual browser window itself.

The browser closes fine, not problem. My issue are the remaining geckodriver processes that are now not bound to anything and keep adding up with each new browser window open.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZLkzydLW0O6iGE0rXX%2B3vqJ4Q4O_gUEDGkWtt7Q175%2BF0g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CA%2BFkgXS7P2PQtyUDEks-NQ_%2B32r%2B%2BDQOkWJHv_9zLOACfAJiag%40mail.gmail.com.

[Eugene Shragovich]

And this is exactly my problem :)

driver.quit() is not being executed and exactly as you describe - does not kill the geckodriver.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CA%2BFkgXS7P2PQtyUDEks-NQ_%2B32r%2B%2BDQOkWJHv_9zLOACfAJiag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/5cQPSnkg__g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZL%3DUNGuNuDEHL6Kz-hnWuDbuu6NbZcUFy3Qn1N9tTD7rGg%40mail.gmail.com.

[⇜Krishnan Mahadevan⇝]

Then you should file a bug with a reproducible test that can be executed to recreate the issue.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Thu, Jan 18, 2018 at 2:25 PM, Eugene Shragovich <shrag...@gmail.com> wrote:

And this is exactly my problem :)

driver.quit() is not being executed and exactly as you describe - does not kill the geckodriver.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZL%3DUNGuNuDEHL6Kz-hnWuDbuu6NbZcUFy3Qn1N9tTD7rGg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CA%2BFkgXRv9HPBngWHjXVmec5him%3D1UC23dMYCspcoPW-uVZKqdg%40mail.gmail.com.

[⇜Krishnan Mahadevan⇝]

But remember, even in the case of client crashes (your test case running JVM crashed or was forcibly killed), the JVM that runs the selenium node (this is obviously a different one from the client) will still clean up the browser and the associated server process (when any of the timeout criteria is met).

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Eugene Shragovich]

Thanks,

So do you say that the selenium standalone server process that is running on the node should cleanup the browser and the associated geckodriver instance even if a specific test has failed without performing a preconfigured cleanup sequence?

On 18 January 2018 at 16:58, ⇜Krishnan Mahadevan⇝ <krishnan.ma...@gmail.com> wrote:

But remember, even in the case of client crashes (your test case running JVM crashed or was forcibly killed), the JVM that runs the selenium node (this is obviously a different one from the client) will still clean up the browser and the associated server process (when any of the timeout criteria is met).

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZL%3DyOmRKWcHwMm6bgxjSYiZeu%3DRovEY_QKSgRz_tOPae1w%40mail.gmail.com.

[⇜Krishnan Mahadevan⇝]

Yep. That is what I have been saying all the while :) because its the same code that gets executed when you use either

• FirefoxDriver.quit() [ In cases wherein you dont use the selenium grid but directly work with the implementation ] (or)
  
• RemoteWebDriver.quit() [ In cases wherein you are using the grid. So your test will send RemoteWebDriver.quit() to the hub, the hub forwards it to the node and the node translates it back to FirefoxDriver.quit() on the node]

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Thu, Jan 18, 2018 at 2:33 PM, Eugene Shragovich <shrag...@gmail.com> wrote:

Thanks,

So do you say that the selenium standalone server process that is running on the node should cleanup the browser and the associated geckodriver instance even if a specific test has failed without performing a preconfigured cleanup sequence?

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CANikZL%3DyOmRKWcHwMm6bgxjSYiZeu%3DRovEY_QKSgRz_tOPae1w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CA%2BFkgXTdV70awRoiuueS%2B_X42S_qMqs7Hbj97KnAwc8jSqc5PA%40mail.gmail.com.