Session Timeout due to disabled Browser Developer Tools

[Venkat Burri]

Hi,

Due to policy, browser developer tools(F12) are disabled within our organization. Since the disablement, we running into session timeouts on Selenium node.

We use Selenium Hub-Node architecture where node utilizes Microsoft Edge and Google Chrome Webdrivers to perform automation tests. Hub dashboard shows nodes successfully registering and in healthy state.

When tests are invoked, we see webdriver (on selenium node) starting on a port and invoking browser. Based on the logs, session api is called but the node never establishes a session ID. The invoked browser hangs without going to URL (based on Selenium test) and timesout.

From Grid dashboard, we noticed request dispatched to correct node based on capability but in session tabs it never establishes session. The request queued at node until timeout without executing tests.

Just for testing, we have tried enabling developer tools on browser and ran selenium test. Tests executed without any issues. We could see sessions being established in grid dashboard and progress without any issues.

We did not find in Selenium documentation that it is required to enable developer tools on browsers to perform selenium tests. 

Wondering if there is any other way, where we keep developer tools disabled and still perform selenium tests. We greatly appreciate any help.

Regards

 

[Venkat Burri]

Following up on this... if anyone has insight...

[Krishnan Mahadevan]

Venkat,

Unlike earlier, Selenium doesn't have a direct control over the browser. 

The middleman in this case is the driver binaries (chromedriver, geckodriver, edgedriver, safaridriver) which is maintained by the respective browser vendors themselves.

Just to try out the theory, here's what you can try doing:

1. Start chromedriver in a terminal (Thanks to Selenium Manager, you should most likely find the relevant binary under the "~/.cache/selenium/chromedriver" location)

2. Now open another terminal and do a curl command that looks like below (If you don't have curl, feel free to translate this to a postman call)

curl --location 'localhost:9515/session' \

--header 'Content-Type: application/json; charset=utf-8' \

--data '{

"capabilities": {

"firstMatch": [

{

"browserName": "chrome"

}

]

}

}'

If you see a browser basically open up in this case (the one which has dev tools disabled), then we can confirm that devtools is required for the driver binaries to interact with the browser.

After that maybe you can confirm the with chrome or the firefox project if your hunch is true. Based on the results, you can go back to your organisation and work with the relevant folks to get exception for this policy and bring this back.

I don't think Selenium has anything to do in this case.

[Venkat Burri]

Thanks Krishnan for taking time to provide me with a very insightful response. 

I ran the test and confirmed that the browser opened with "data:," in address bar and with developer tools disabled. I do agree with you that this maybe driver issue as selenium simply uses browser webdriver to invoke browser. I will initiate requests with webdriver/browser vendors.

Do you know the detailed steps happen between when a selenium test invoked to grid and browser opened to perform functional test?

Does it help in anyway to create a custom proxy to monitor calls to nodes? 

Regards

Venkat

[Krishnan Mahadevan]

Venkat,

Just to be explicitly clear.

Scenario1: Dev Tools disabled, chrome driver started, fired curl new session command. Result - Browser Does not open

Scenario2: Dev Tools enabled, chrome driver started, fired curl new session command. Result - Browser opens

Is this what you are experiencing?

Now coming to the part of tracing the request.

You should be able to see all of this in action when you do the following:

1. Spin up a hub in your local machine.

2. Spin up a node with remote debugging enabled (You can refer to the answers of this question to learn how to do this: https://stackoverflow.com/q/975271 )

3. Setup debug points in your IntelliJ project (you can create a sample project that depends on org.seleniumhq.selenium:selenium-grid) from org.openqa.selenium.grid.node.local.LocalNode#newSession

3. Fire a curl command for a new session against the hub.

4. You should be able to walk through the calls and understand the end to end call invocation.

I am not sure what the custom proxy is going to be doing to monitor calls. As I mentioned earlier, if the driver binary has an undocumented dependency on Developer tools being enabled for it to be able to interact with the browser, then that’s the part that needs to be looked at.

The other option you can consider here is to setup a hub which has docker backed nodes. That way, the hub would forward the commands to the nodes (which are now going to come up as on demand docker containers) wherein your tests are guaranteed to run irrespective of whatever is your org policy, because the browser opens up inside the container and policy changes don’t affect docker containers (atleast not the best of my knowledge). 

This way it will be easier for you to keep your tests happy and also adhere to whatever policies are set by your organisation.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/5448c935-9d9e-431e-be2d-1d8af38ddad7n%40googlegroups.com.

[Venkat Burri]

Hi Krishnan

Scenario1: Dev Tools disabled, chrome driver started, fired curl new session command. Result - Browser does open with "data;," in address bar. Curl/postman call returns with a valid session ID.

Scenario2: Dev Tools enabled, chrome driver started, fired curl new session command. Result - Browser opens with "data;," in address bar. Curl/postman call times out.

Thanks for the suggestion with node remote debugging. I will try that today. 

I am also working with browser vendor to get a confirmation on developer tool dependency for webdriver. Will update with results here.

Regards

Venkat

On Monday, May 13, 2024 at 12:17:35 AM UTC-4 Krishnan Mahadevan wrote:

Venkat,

Just to be explicitly clear.

Scenario1: Dev Tools disabled, chrome driver started, fired curl new session command. Result - Browser Does not open

Scenario2: Dev Tools enabled, chrome driver started, fired curl new session command. Result - Browser opens

Is this what you are experiencing?

Now coming to the part of tracing the request.

You should be able to see all of this in action when you do the following:

1. Spin up a hub in your local machine.

2. Spin up a node with remote debugging enabled (You can refer to the answers of this question to learn how to do this: https://stackoverflow.com/q/975271 )

3. Setup debug points in your IntelliJ project (you can create a sample enabled project that depends on org.seleniumhq.selenium:selenium-grid) from org.openqa.selenium.grid.node.local.LocalNode#newSession

[Venkat Burri]

Final Resolution:

On testing different scenarios and based on documentation, it is found that session timeout is caused by Browser developer tools disablement. Microsoft documentation confirmed this as well

https://learn.microsoft.com/en-us/microsoft-edge/webdriver-chromium/?tabs=c-sharp

Developer Tools Availability policy

If your IT admin has set the DeveloperToolsAvailability policy to 2, Microsoft Edge WebDriver is blocked from driving Microsoft Edge, because the driver uses Microsoft Edge DevTools. To automate Microsoft Edge, make sure the DeveloperToolsAvailability policy is set to 0 or 1.

Thanks 
Venkat