Tests are failing randomly on CI - testng/linux/docker

[garvitag...@gmail.com]

Hi,

I am facing strange issue, my test run perfectly fine on local everytime. I tried on others system as well, windows/linux both. Everytime it works fine. But on CI, everytime  a new error comes. Sometimes its on wait, sometime son click, sometimes stale element.  

As CI runs in docker headless mode, i really dont know how to debug this thing. All this started after updating chrome to 66 (thats my assumption). 

I am using :

webdriver 3.11

testng 6.13

allure 2.35

chromedriver 2.38

chrome 66

Can someone help me with ways to debug this . There is no consistent error, its failing anywhere. I tried running headless mode in local, again no issue. I have mostly used explicit waits in framework, there is no hard coded/thread.sleep that can cause inconsistent behaviour on CI.

Thanks !!

[David]

When you run locally, are you running using docker or just on your localhost OS in headless mode? Hopefully the former to better try reproduce the issue.

[garvitag...@gmail.com]

David,

I am running without docker on local. on your localhost OS in headless mode. Site is not accessible publicly thats why i was not able to provide code to reproduce issue. 

Thanks !! 

[David]

It could be a docker issue, so I suggest you try running test via the docker setup locally. If you can't run docker on your localhost, try finding a machine that can besides CI server/service.

Also for the CI issue, also recommend you run it in CI in debug mode if the CI has such option. e.g. if using CircleCI, you can SSH into the CI job container and try things out to debug rather than just watch the build progress, that would help to see if it just fails during normal CI or if it also fails in debug mode too (which would be a more serious issue). If it fails in normal CI run only, then that's likely a timing, race condition issue.

The local docker debugging is to determine if issue is with docker setup or with docker within the CI environment