ATH - Selenium times out trying to wait until all the plugins get installed

[Dhivyaa Nandakumar]

  I am using ATH to test a jenkins pipeline library.

  It looks like selenium timesout trying to wait until all the plugins are installed.

  The piece of code is in https://github.com/jenkinsci/acceptance-test-harness/blob/master/src/main/java/org/jenkinsci/test/acceptance/po/UpdateCenter.java
 
  The tests fail most of the time due to this time out exception.

Stack trace -

Cleaning up temporary JENKINS_HOME failed, retrying in 5 sec.

org.openqa.selenium.TimeoutException: Timed out after 60 seconds: not contains element that matches By.xpath: //*[@id='log']//*[contains(.,'Pending') or contains(.,'Installing')]. was <org.openqa.selenium.support.events.EventFiringWebDriver@25a94b55>
Build info: version: '2.53.0', revision: '35ae25b1534ae328c771e0856c93e187490ca824', time: '2016-03-15 10:43:46'

    at org.openqa.selenium.support.ui.FluentWait.timeoutException(FluentWait.java:292)
    at org.jenkinsci.test.acceptance.junit.Wait.timeoutException(Wait.java:159)
    at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:261)
    at org.jenkinsci.test.acceptance.junit.Wait.until(Wait.java:144)
    at org.jenkinsci.test.acceptance.junit.Wait.until(Wait.java:110)
    at org.jenkinsci.test.acceptance.po.CapybaraPortingLayerImpl.waitFor(CapybaraPortingLayerImpl.java:192)
    at org.jenkinsci.test.acceptance.po.UpdateCenter.waitForInstallationToComplete(UpdateCenter.java:74)
    at org.jenkinsci.test.acceptance.po.PluginManager.installPlugins(PluginManager.java:260)
    at org.jenkinsci.test.acceptance.junit.WithPlugins$RuleImpl$1.installPlugins(WithPlugins.java:177)
    at org.jenkinsci.test.acceptance.junit.WithPlugins$RuleImpl$1.evaluate(WithPlugins.java:130)
    at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
    at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1$2$1.evaluate(JenkinsAcceptanceTestRule.java:175)
    at org.jenkinsci.test.acceptance.junit.FilterRule$1.evaluate(FilterRule.java:63)
    at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
    at org.jenkinsci.test.acceptance.junit.JenkinsAcceptanceTestRule$1.evaluate(JenkinsAcceptanceTestRule.java:60)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

[Ullrich Hafner]

You should never start ATH tests without a preconfigured Jenkins, this makes the tests too slow. 

See description of JUT and PLUGINS_DIR in the doc/ folder.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/89e06a0d-2f23-4df0-8200-37a548e94ac9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Oliver Gondža]

What ATH version are you running? The code is expected to timeout after
60 seconds[1] and not 5, as in your case. I am speculating if the fact
you are invoking the tests from IDEA can not be the cause but I do not
see a reason why it would matter.

Also, have you seen
https://github.com/jenkinsci/acceptance-test-harness/blob/master/docs/INVESTIGATION.md?

[1]
https://github.com/jenkinsci/acceptance-test-harness/blob/95202f9bdf020169695252218624d696185b30bc/src/main/java/org/jenkinsci/test/acceptance/po/UpdateCenter.java#L74

--
oliver

[Dhivyaa Nandakumar]

I am using the latest version of ATH. 

Yes, it does wait for 60 seconds and times out after. Apparently, my tests take more than 60 seconds to install all the plugins.

[Dhivyaa Nandakumar]

I have tried running the tests inside a docker container as well. Selenium times out there too.

I am using a bunch of plugins in my tests. The fact that it takes more time(more than 60 seconds) to download and install all the plugins for my tests make selenium timeout at line 74 in UpdateCentre.java [https://github.com/jenkinsci/acceptance-test-harness/blob/95202f9bdf020169695252218624d696185b30bc/src/main/java/org/jenkinsci/test/acceptance/po/UpdateCenter.java#L74 ]. I am wondering if there is a reason to limit the timeout to 60 seconds. 

[Oliver Gondža]

On 2017-10-10 22:10, Dhivyaa Nandakumar wrote:
> I have tried running the tests inside a docker container as well.
> Selenium times out there too.
>
>
> I am using a bunch of plugins in my tests. The fact that it takes more
> time(more than 60 seconds) to download and install all the plugins for
> my tests make selenium timeout at line 74 in UpdateCentre.java
> [https://github.com/jenkinsci/acceptance-test-harness/blob/95202f9bdf020169695252218624d696185b30bc/src/main/java/org/jenkinsci/test/acceptance/po/UpdateCenter.java#L74

> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fjenkinsci%2Facceptance-test-harness%2Fblob%2F95202f9bdf020169695252218624d696185b30bc%2Fsrc%2Fmain%2Fjava%2Forg%2Fjenkinsci%2Ftest%2Facceptance%2Fpo%2FUpdateCenter.java%23L74&sa=D&sntz=1&usg=AFQjCNG1CHQZ26KNGUM76wGCUA2Kz8CfNg> ].

> I am wondering if there is a reason to limit the timeout to 60 seconds.

There are several reasons this might not work for you.

- Your "setup" is just slower compared to our reference infra. In that
case, you might want to slow down the suite[1] as well.
- It is more likely, however, that you are observing some sort of
failure that manifests this way. Have you consulted the diagnostics
directory as I suggested? What is Jenkins doing at the time of failure
(threaddump.log)? What does the page contains? If you run the tests with
`INTERACTIVE=true` and wait a little longer, will the element come to
existence?

[1]
https://github.com/jenkinsci/acceptance-test-harness/blob/47ae4d5cd6e1c17308c1dc2e28ee6b3b11bc3a0d/src/main/java/org/jenkinsci/test/acceptance/utils/ElasticTime.java#L42-L47

--
oliver

[Dhivyaa Nandakumar]

I have attached the jenkins thread dump.

With  "INTERATIVE = true" environment variable, the browser was still open with the timeout exception. Plugin installation continued but the test execution did not resume.

I did try increasing the selenium timeout value in [https://github.com/jenkinsci/acceptance-test-harness/blob/95202f9bdf020169695252218624d696185b30bc/src/main/java/org/jenkinsci/test/acceptance/po/UpdateCenter.java#L74 ]. This works and all my tests pass. I feel it is the network connectivity which leads to timeout exception. Is there a way to make this timeout number a variable using command line argument with a default of 60 seconds?