New open-source Selenium library makes StaleElementReferenceException a thing of the past

211 views
Skip to first unread message

Scott Babcock

unread,
Nov 30, 2017, 8:19:16 PM11/30/17
to Selenium Users
https://github.com/Nordstrom/Selenium-Foundation

Selenium Foundation is an automation framework designed to extend and enhance the capabilities provided by Selenium 2.0 (WebDriver).

This library provides a complete set of base classes for building well-factored page models. This includes page components and frames. Selenium Foundation allows you to focus on modeling your application (instead of managing which window or frame the driver is addressing) by handling all driver targeting for you. You'll never see driver.switchTo(...) in page model automation built with Selenium Foundation, because the framework automatically ensures that the driver is addressing the window or frame associated with each page model method before it's invoked.

Page classes can be explicitly associated with web application paths through the @PageUrl annotation. These associations can be declared as either fixed paths or patterns, and these declarations are used by Selenium Foundation to verify landing page paths at page transitions. You can also perform direct navigation to web application paths associated with page classes through the @PageUrl annotation.

Selenium Foundation also includes collection classes (ComponentList, ComponentMap, FrameList, and FrameMap) that enable you to define collections of components for your page models. For example, you can define a SearchResultTile component and include a map of these tiles keyed by product ID in your SearchResultsPage class. Selenium Foundation collections are lazy-initialized automatically - the composition of the collection is determined when it's instantiated, but each item in the collection is only populated when it's explicitly referenced.

One of the most impactful features of Selenium Foundation saves your automation from the dreaded StaleElementReferenceException failure. Web element search operations performed within the Selenium Foundation framework return enhanced references, which retain all of the parameters needed to re-acquire the reference if it goes stale. Every web element method call is guarded by an automatic recovery feature. If a reference goes stale, Selenium Foundation re-acquires the reference and re-issues the web element method call that encountered the exception. Your automation continues on normally, blissfully unaware of the disaster that was averted.

Another useful extension provided by Selenium Foundation is the optional element. This feature enables you to model elements that only exist on the page under specific conditions. For example, you can model an error message that only exists when a form is submitted with no value in a required field. Determining if the element exists is as easy as calling the hasReference() method of the optional element object.

Selenium Foundation automatically synchronizes your automation with ordinary page transitions, ensuring that tests don't get tripped up by application hesitation. Synchronizing your automation with dynamic content creation is easily done by implementing a simple interface (DetectsLoadCompletion). This greatly simplifies the modeling of single-page applications and pages rendered with dynamic content loading.

To avoid divergent behavior between local and remote execution, Selenium Foundation acquires driver sessions for local runs from a local instance of Selenium Grid. In addition to eliminating ordinary behavioral differences, this strategy provides two major benefits:
  1. Adding support for a new driver is a simple configuration change - No need to crack open the code!
  2. You get explicit control over the maximum number of concurrent sessions, so you can run your tests in parallel without overtaxing your system.
Drivers allocated for per-test configuration setup methods (i.e. - @BeforeMethod) are automatically handed off to the tests for which configuration is being performed. Drivers allocated for tests are automatically handed off to per-test configuration cleanup methods (i.e. - @AfterMethod).  This hand-off behavior enables you to implement generic setup and cleanup processing that interacts with your application under test.

To assist in root-cause analysis, Selenium Foundation automatically captures a screenshot and page source for each failed test. By using the ReporterAppender, the log output of each TestNG test is captured as part of the test result object. This information is automatically shown on test result pages in Jenkins. No more digging through intermingled output in console logs!

Selenium Foundation includes support for both TestNG and JUnit. Feature parity is enabled by several core abstractions, and through features provided by the TestNG Foundation and JUnit Foundation libraries.

Jim Evans

unread,
Dec 1, 2017, 6:58:08 AM12/1/17
to Selenium Users
So you’ve decided not to support recent and future versions of Firefox in this framework? By emphasizing only the 2.x branch of Selenium, and not allowing any support for 3.x, that’s what you’re implying.

Scott Babcock

unread,
Dec 1, 2017, 4:43:42 PM12/1/17
to Selenium Users
I have no philosophical objection to Selenium 3.x. However, I need to actually get work done. In my experience, Selenium 3.x is still pretty rough. It's still missing fundamental features and suffers from stability issues. Once the new API stabilizes, I'll be happy to upgrade.

Jim Evans

unread,
Dec 1, 2017, 9:10:16 PM12/1/17
to Selenium Users
I disagree with your assessment. I’d like to know what “fundamental features” it lacks, and what “stability issues” you feel it suffers from.

Scott Babcock

unread,
Dec 1, 2017, 10:01:56 PM12/1/17
to Selenium Users
It's an open-source project. Clone the repo, upgrade to the latest Selenium, and see how it flies.

Jim Evans

unread,
Dec 2, 2017, 9:54:51 PM12/2/17
to Selenium Users
I’m sorry, I don’t follow. Which repo should I clone to see what features Selenium 3.x lacks? The Selenium repo?

Krishnan Mahadevan

unread,
Dec 3, 2017, 1:53:24 AM12/3/17
to seleniu...@googlegroups.com

Scott,

 

Jim Evans is the Lead Developer for the IE Bindings and he is part of the Core Selenium Development community.

It’s only fair to say that he would know the Selenium 3 bindings and its short-comings/pitfalls/instabilities (if any) more than any of us.

 

So it would be great, if you could help call out the specifics of what you are highlighting as:

 

  1. Fundamental features that are missing
  2. Stability issues.

 

Thanks & Regards

Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

On 12/3/17, 8:25 AM, "Jim Evans" <seleniu...@googlegroups.com on behalf of james.h....@gmail.com> wrote:

 

    I’m sorry, I don’t follow. Which repo should I clone to see what features Selenium 3.x lacks? The Selenium repo?

   

    --

    You received this message because you are subscribed to the Google Groups "Selenium Users" group.

    To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.

    To post to this group, send email to seleniu...@googlegroups.com.

    To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/91c793b3-3f0d-4535-8557-b7ef629c665f%40googlegroups.com.

    For more options, visit https://groups.google.com/d/optout.

   

Scott Babcock

unread,
Dec 3, 2017, 7:35:18 AM12/3/17
to Selenium Users
My apologies for the tone of my response. At the time I started working on the Selenium Foundation library, the current release of Selenium was 3.3.1. In this release, hangs and crashes were common, so I stuck with Selenium 2.53.0. Lack of backward compatibility in the Selenium 3 Grid API made it difficult to try new releases as they were deployed, so it's been a while since I evaluated Selenium 3 myself. Periodic review of open issues made me reluctant to make the switch.

I spent a few hours today to create a Selenium 3 Foundation project: https://github.com/Nordstrom/Selenium3-Foundation
Release 1.0.0 of this project is available now from Maven Central. I'll keep the Selenium 2 and Selenium 3 projects in sync.

Scott Babcock

unread,
Dec 3, 2017, 8:06:38 PM12/3/17
to Selenium Users
I'll also take the time in the next week or so so factor these two parallel projects into a single hierarchical project, to eliminate the unnecessary duplication. I also need to upgrade the Guava dependency in TestNG Foundation to eliminate a transitive dependency issue with URLChecker.

Scott Babcock

unread,
Dec 3, 2017, 9:16:06 PM12/3/17
to Selenium Users
By the way, the latest build of standalone server has a re-occurrence of this issue: https://github.com/SeleniumHQ/selenium/issues/3534
I rely on PhantomJS for headless operation, so this is a blocking issue for me.

Krishnan Mahadevan

unread,
Dec 3, 2017, 9:43:27 PM12/3/17
to seleniu...@googlegroups.com

Scott,

 

I don’t think the issue is with Selenium here. Piggy backing on Selenium’s internal APIs always have this risk. Ideally speaking, if one is using any of these APIs then they should pay attention to the deprecation messages and keep up with the Selenium versions so that one doesn’t run into these sort of issues.

 

Its perhaps due to GhostDriver working with an older version of Selenium (3.4.0) and relying on a deprecated/deleted internal API (Commandline#find())

 

You could try sending a pull request to : https://github.com/codeborne/ghostdriver

 

Until that PR gets merged, as a short term solution, you could try

 

  1. duplicating the Commandline class into your project from Selenium 3.4.0 maintaining the same package structure and thus trick Java into resolving the class with the method available as a temporary solution.
  2. Try re-ordering your Maven dependencies and thus cause Selenium 3.4.0 from within ghostdriver to be resolved ahead of your selenium dependency (you may have to take care to ensure that you reduce the impact to the artifact in which CommandLine resides viz., selenium-remote-driver). Am not sure how much this would work though.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

Scott Babcock

unread,
Dec 4, 2017, 12:05:15 AM12/4/17
to Selenium Users
The issue with phantomjs is that the PhantomJSDriver class can't be found. This used to be part of the standalone server JAR, but has apparently been removed. I can't find any reference to this in the release notes, but these are only available through release 3.5.3

⇜Krishnan Mahadevan⇝

unread,
Dec 4, 2017, 12:40:25 AM12/4/17
to Selenium Users
Sounds like it was removed.


So you may have to perhaps explicitly add a dependency to move forward.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/

On Mon, Dec 4, 2017 at 10:35 AM, Scott Babcock <sco...@gmail.com> wrote:
The issue with phantomjs is that the PhantomJSDriver class can't be found. This used to be part of the standalone server JAR, but has apparently been removed. I can't find any reference to this in the release notes, but these are only available through release 3.5.3
--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/37a02969-d521-439f-88f7-d4c688aff41f%40googlegroups.com.

Jim Evans

unread,
Dec 4, 2017, 2:45:55 AM12/4/17
to Selenium Users
PhantomJS support has been largely deprecated. Given that the PhantomJS project has been mostly abandoned, and that it was never a good analogue for any browser that a user would actually use to browse a site, this doesn’t seem to be an unreasonable decision. Additionally, since both Firefox and Chrome have vendor-supported headless modes, there are better (or at least more accurate to the end-user browsing experience) options.

Scott Babcock

unread,
Dec 4, 2017, 2:20:47 PM12/4/17
to Selenium Users
As far as I can tell, there's currently no support to run either Chrome or Firefox headless from Selenium Grid. This is a requirement for me. Beyond this, I'm not sure how I would specify both headless and visible instances of the same browser on a single Selenium Grid node. The correlation between browser name a driver appears to be hard-coded, which would seem to preclude specification of headless and visible capabilities simultaneously.

Jim Evans

unread,
Dec 4, 2017, 4:28:45 PM12/4/17
to Selenium Users
Really? Something like the following doesn't work?

// WARNING AND DISCLAIMER: The following code was written without
// the benefit of an IDE, and was done strictly from memory. It
// may not be entirely semantically or syntactically correct
// without modification.
// Given that ChromeOptions implements Capabilities, it should
// be passable to the RemoteWebDriver constructor. Note that
// flipping between headless and non-headless is as simple as
// changing the boolean value to the setHeadless() call. It
// is set on a per-session basis, not per-browser.
ChromeOptions options = new ChromeOptions();
options.setHeadless(true);
WebDriver driver = new RemoteWebDriver("http://path/to/grid/hub", options);

Message has been deleted

Scott Babcock

unread,
Dec 5, 2017, 3:01:46 PM12/5/17
to Selenium Users
It appears that my difficulties with Chrome were related to a bug that was resolved by Selenium 3.8.1. Here's the JSON I used:

{
  "browserName": "chrome",
  "chromeOptions": { 
    "args": [ "--disable-infobars", "--headless", "--disable-gpu" ], 
    "prefs": { "credentials_enable_service": false } 
  } 
}

I also added the new dependency that provides PhantomJSDriver, which got that working again.
The only thing left to figure out is Firefox. It's extremely slow, and I haven't gotten it to run headless yet. I'll keep working with this to see if I can determine the cause of these problems.

On Monday, December 4, 2017 at 1:32:04 PM UTC-8, Scott Babcock wrote:

Scott Babcock

unread,
Jan 28, 2018, 1:25:28 PM1/28/18
to Selenium Users
I just finished consolidating the Selenium 3 variant of Selenium Foundation back into the main project. I ran out of ideas trying to get Maven to produce and publish both configurations from a single project, so I transitioned to Gradle. I still have a few wrinkles to iron out with the release process, but the first fully-functional release from this combined project is version 14.1.14.

To differentiate between the two variants, I chose the strategy that Google used with Guava - designating the variant with a suffix on the version.
* Version `14.1.14-s2` supports Selenium 2
* Version `14.1.14-s3` supports Selenium 3.

Scott Babcock

unread,
Feb 7, 2018, 10:13:54 PM2/7/18
to Selenium Users
I think all of the wrinkles in my Gradle project have been resolved. The latest version (14.1.15) has been released. The POM included with this release is still the one built by the Maven plugin, but the next release will have a custom-built POM that's actually capable of building the project.
Reply all
Reply to author
Forward
0 new messages