How to figure out why robot test run jams

[Hi-Fi]

Hi,

I have quite "interesting" problem with Robot framework. We have CI-server (Linux), which has Jenkins and which runs tests (and builds) with Maven.

Test run has been working fine for ~3 months, but has now started to do very strange jams; run just gets stuck, and nothing I have figured out to check tells why.

I tried increase the loglevel in pybot, but it didn't provided any help.

Things I tested:

FAIL: Running tests normally, so maven makes build, starts local server (Jetty), and starts tests (using robot-maven-plugin from org.codehaus.mojo) (Using selenium2library, so can't use plugins that start tests directly).

SUCCESS: Checking application with other browser when jam occures, and it's working fine

SUCCESS: Starting server in one console, and tests in other

SUCCESS: Starting server in CI-server, and running tests from own laptop.

So server isn't jamming, pybot somehow is.

Any ideas how to fix this/how to find out what's the problem?

Python 2.7.3 (default, Jun 15 2012, 15:28:38)

[Hi-Fi]

And one more note: run works fine with other Windows/Linux computers. 

[Hi-Fi]

Oh, and about loglevel -paramater: It doesn't help, because when tests execution stops, log writing is stopped immediately. With default level this means that last line was "<arg>${".

Process itself isn't killed; it just "jams" (and lives as long as it's either manually killed or run manually interrupted).

[Pekka Klärck]

2012/9/20 Hi-Fi <juho.s...@gmail.com>:

> Oh, and about loglevel -paramater: It doesn't help, because when tests
> execution stops, log writing is stopped immediately. With default level this
> means that last line was "<arg>${".

Writing to output file is buffered, which means that OS typically
doesn't immediately write text written there to the disk. To get a
more accurate information where the execution was when it hanged, you
could try enabling --debugfile. That file is flushed so everything
written there ought to end up to the disk immediately.

Cheers,
.peke
--
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

[Hi-Fi]

Thank you very much. Debug log just made more questions, because the part where test jammed was login, which is used in every test at least once.

Seems that there's some problem with Selenium2library's keywords when clicking element; the "don't wait" argument is removed, and most time there needs to be explicit wait in test case. But for some reason now it seems, that test jams when clicking button/element.

Maybe only way to figure this out is to add some printing directly to webdriver itself? 

[Ed Manlove]

On 09/21/2012 01:36 AM, Hi-Fi wrote:

On Thursday, September 20, 2012 6:37:14 PM UTC+3, Pekka Klärck wrote:

Thank you very much. Debug log just made more questions, because the part where test jammed was login, which is used in every test at least once.

The thing that helped was to change "Run keyword and ignore error" keyword to "Get matching xpath count" (because most of those were just checking some elements). No idea why, and no idea if the problem occured in Python, Robot framework, selenium2library or somewhere else.  

I've added some notes to the Selenium2Library documentation which outlines how to write some pure Selenium python unittest test cases so one can differentiate whether an issue is within Selenium2Library or Selenium. Those notes can be found at

  https://github.com/rtomac/robotframework-selenium2library/blob/master/BUILD.rst#testing-third-party-packages

and will be found in the upcoming 1.1 release of Selenium2Library. I had a couple of issues for which I used this procedure leading to a better understanding of where my problems were.  Hope this helps.

Ed

[Hi-Fi]

Thanks, I have to try that. 

I tried with own PHP-page, and it seems that Timeout value isn't affecting to at least "Open browser", "Go to" and "click link" keywords. The original problem is (I think) that for some reason in test either browser or server has small jam, which wouldn't prevent running test suite to end. But now it does, because Selenium seems to be waiting answer 'til eternity.

Test page was just couple of echos, and 4 second sleep. Selenium2library timeout was set to 2 seconds, so test should have failed, but it passed.

[Hi-Fi]

Tried that unit testing 

import unittest

from selenium import webdriver

class ExecuteJavascriptTestCase(unittest.TestCase):

    def setUp(self):

        self.driver = webdriver.Firefox()

        self.driver.implicitly_wait(2) # seconds

    def test_exe_javascript(self):

        driver = self.driver

        driver.get("http://localhost/test")

        print('Finished')

    def tearDown(self):

        self.driver.close()

if __name__ == "__main__":

    unittest.main()

And result was, that test passed even the page that was loaded had sleep of 4 seconds. So opening page really doesn't seem to care about timeouts.

[Hi-Fi]

This seems to be quite old known issue in Selenium:  http://code.google.com/p/selenium/issues/detail?id=687

In old seleniums this kind of problem would have been possible to bypass by using "don't wait", and then giving some wait that knows timeouts.

[Hi-Fi]

Just a quick note if somebody else experiences same kind of problem: For me it helped when I "cycled" (closed and re-opened) browser at the beginning of the jamming test suite. Now whole set is going through fine.