Upgrading from 0.10.0: all wait-less magic has gone. Regression or feature?

[Andrey Hitrin]

Hi!

In our company we have a reasonably large suite of acceptance tests (several hundred tests) written in Geb. Currently, we're using Geb 0.10.0, Selenium 2.46.0 and Firefox (both local and remote).

The problem appears when we try to upgrade our dependencies. Namely, I try to upgrade Geb to the newest 0.13.1 without touching other dependencies.

It looks like WebDriver starts to act way more eager than before, throwing away some previously existing waits. This causes our tests to fail in many random places unless we put a lot of waitFor into it. For example, when working with dialogs:

// Works fine with Geb 0.10.0, but not in the any newer version
browser.withNewWindow({ someButton.click() }) {
    $('input#title').value(title)
    ...
}

After upgrading to the newer versions this code may fail because WebDriver may start to act on the dialog before page actually loads. When I put simple assert !$('body').empty into the beggining of the window closure, it fails randomly (but never fails in Geb 0.10.0). So, in order to fix our tests we have to add a lot of waits, like that one:

browser.withNewWindow({ someButton.click() }) {
    waitFor { $('input#title').displayed }      // or, for example, waitFor{ !$('body').empty }
    $('input#title').value(title)
    ...
}

But we have a lot of dialogs in our system under test, so amount of code to change is really large. That's pity.

Situation is a little more strange because I cannot reproduce this problem in virto. When I just save HTML page of our application and run it under Python SimpleHTTPServer, synchronisation issue on the dialog disappears. But in our application (Spring web app running under Tomcat) such failures appear very frequently.

Seems like we have to stay on 0.10.0 for now, but it's unclear for me whether the situation with waits will change in future. It's also unclear whether it's a bug of Geb or not.

May I ask you for help?

[Andrey Hitrin]

Even more, sometimes tests fail even on moving to another page:

$('a', text: 'click me').click(NextPage)

Works perfect in Geb 0.10.0, but in 0.13.1 it may cause UnexpectedPageException:

geb.error.UnexpectedPageException: At checker page verification failed for page blah.meh.NextPage
 at geb.navigator.NonEmptyNavigator.click(NonEmptyNavigator.groovy:492)

 at geb.navigator.NonEmptyNavigator.click(NonEmptyNavigator.groovy:469)
 at ...

Meanwhile error screenshot shows us that we're on the NextPage, as expected.

[Marcin Erdmann]

Hi Andrey,

What you're seeing is most probably down to performance improvements in Geb which were introduced in 0.12.x and 0.13.x and which boil down to Geb not executing some unnecessary commands that it did in the past. I can point you to issues and commits that relate to that if you want me to.

What adds to the situation you're facing is the fact that Firefox driver has historically been flakey when it comes to waiting for the page to load - it should wait until any callbacks registered on page load execute but it doesn't sometimes it will return control even before any html is loaded. To aid that at some point we introduced the atCheckWaiting config option (http://www.gebish.org/manual/current/#at-check-waiting). I would suggest you configure it to wait for up to a second if you're using Firefox as your driver. We are at my current project and without it we would almost always have a test that fails when switching pages and running at checkers.

A solution to your other issue would be to model your pop ups as pages with at checkers and use the page option of withNewWindow() method (http://www.gebish.org/manual/current/#code-page-code-2). Having atCheckWaiting enabled as well would mean that the Geb would ensure that the page has loaded in the new window before handing control back to the closure you pass as the last argument in your withNewWindowCalls.

So to confirm - this is not a bug in Geb as Geb always interacts with the browser via WebDriver and this includes navigating to urls. It's just that making Geb this bit faster exposed situations where WebDriver implementations should wait for the page to load but they don't. I understand that this might be painful but this is how it is, we shouldn't be slowing things down to avoid timing issues like that.

It might seem like a lot of work to upgrade and fix the issues you're seeing but I would advise you to do so if only you can afford the time. You might hit a blocking issue one day which will mean that you will be forced to upgrade over many versions at the worst possible moment and it will take more time the longer you wait. I've learned that being on the latest versions of your core tools always pays off.

Marcin

четверг, 7 апреля 2016 г., 15:39:40 UTC+5 пользователь Andrey Hitrin написал:

--
You received this message because you are subscribed to the Google Groups "Geb User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+u...@googlegroups.com.
To post to this group, send email to geb-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/dba3e2bb-a38b-481f-b416-dca739660ae5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Andrey Hitrin]

Marcin, thank you a lot for your quick and detailed response! I'm so sorry I couldn't be as fast as you...

Your suggestions helped me a lot, and after two or three weeks of hard work I had finally achieved a 100% stability of our tests using the 0.13.1 Geb release. This was very great!

What changes we had to do:

1. Add new options to the GebConfig.groovy

  atCheckWaiting = true
  baseNavigatorWaiting = true

2. Add waitFor steps after closing of new windows (maybe it's needed because of behavior of SUT). Something like this:

  browser.withNewWindow({addItemButton.click()}) {
     ... (setup item and press 'save')
  }
  waitFor {itemsList.find('td')*.text().contains(itemName)}

3. Add new page object for all dialogs (this advice was really helpful, thank you!). In order to minimize code changes (we have a LOT of dialogs) this page object is used be deafult as options.page right in the overwritten Browser.withNewWindow method (gods bless Groovy metaprogramming!) Of course, it looks like ugly hack, but we already had to overwrite this method for some reasons. So this change may be counted as "small".

4. Explicitly define target page for all clicks that cause page refresh:

  someLink.click()
  # becomes
  someLink.click(AwesomePage)
  # or for clicks inside page object methods that do not change the page
  someLink.click(this)

This change was one of the biggest (since we have a lot of clicks too!). But at least it improves code readability :)

5. We also had to give up using long page lists in the to: content option, like this:

  class LoginPage extends Page {
    static content = {
      loginButton(to: [UserPage, SuperUserPage, NoRightsPage, ...]
    }
  }

All page objects in this list are checked sequentially, but only one check could be successful. With 5 pages in the list and 10 seconds waiting in the GebConfig.groovy (our settings) this means that we have to wait up to 40-50 seconds after each login until the test continues to work. I've tried to overwrite the way Geb looks for pages here into something like "run all checkers simultaneously and wait until one of them succeeds" but had no success (I didn't try too hard, though). So we've had to explicitly define expected target page for the login button.

All this work has been done a few month ago, but I think you're still interested in feedback. Our experience in working with the new Geb version shows that it's stable, useful and still rocks. Sadly, we haven't got any significant improvement in test speed after the transition, but I think the cause lies in the speed of our SUT. When (for example) 90% of click() time is server response time and 10% is WebDriver+Geb time, it's hard to see Geb improvements :) But it doesn't mean these improvements are useless! Thank you for this work anyway!

четверг, 7 апреля 2016 г., 15:39:40 UTC+5 пользователь Andrey Hitrin написал:

Hi!

[Marcin Erdmann]

Thanks for writing up your experience, this might be useful for others in the future.

Marcin

--

You received this message because you are subscribed to the Google Groups "Geb User Mailing List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to geb-user+unsubscribe@googlegroups.com.

To post to this group, send email to geb-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/d06cee4e-ece3-4662-a494-5ee4b16aac01%40googlegroups.com.

[Raja Rajan]

Erdy , 

How do we configure atCheckWaiting configuration to one second , as you mentioned.Please help

[antony...@vendigo.com]

Hi Raja,

It's documented here: http://www.gebish.org/manual/current/#page-level-at-check-waiting

In a pinch, either in your spec or in GebConfig, you can simply do:

atCheckWaiting = 1

[Raja Rajan]

Thank you very much..What is that 1 mean to ?Is that mean 1 sec?

[Antony Jones]

Hi Raja,

It's clearly laid out in the documentation. I strongly suggest you click the link and have a quick read of it. The documentation is shorter than this email in fact!

In summary though, any number will be used as 'seconds to wait', strings reference wait configurations, and true/false enable/disable defaults respectively.

Antony

You received this message because you are subscribed to a topic in the Google Groups "Geb User Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/geb-user/6qpz3hfyBpg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to geb-user+u...@googlegroups.com.

To post to this group, send email to geb-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/geb-user/72136705-677c-4295-83a4-f5ee23de87d4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Antony Jones | Chief Technical Architect

Vendigo Finance Regional | www.vendigo.com | antony...@vendigo.com 

[Raja Rajan]

Thanks a lot for the answer :)