Shadow DOM and the proposed webdriver spec

[Zachary Conrad]

Hello selenium-developers,

I am currently working on a single page web application that uses shadow DOM very extensively.  To allow automated testing of this application, I am working on adding shadow DOM support to chromedriver, based on the working draft at http://www.w3.org/TR/webdriver/.  I understand that the draft is a work in progress, but I have a few concerns about the draft and hoping to spark some discussion about them.

Concern #1: The proposal treats shadow DOM a lot like an iframe, and using iframes with webdriver sucks.

You must call switchToSubTree(element, index) to interact with anything in a shadow DOM, very similar to how you must call switchToFrame(frame) to interact with an iframe.  I understand there are probably technical and security-related reasons this is required for interacting with an iframe.  This is not necessarily true for items in a shadow DOM, which are all controlled by the domain hosting them.

Concern #2: It is extremely likely that something in the shadow DOM will affect something outside the shadow DOM.  Unlike an iframe.

A classic example of an iframe would be an embedded '+1' or 'like' button.  Clicking the button actually has no effect on anything outside the iframe.  This is not necessarily true for something in a shadow DOM.

Here is an example from the single page web application I want to test.  The application has a table component implemented with shadow DOM and a dropdown menu component (with filters for the table) also implemented with shadow DOM.  Both these subcomponents are combined into a TableWithFilter component that has its own shadow DOM.  Now suppose I want to test that a certain filter has 5 results.

table_with_filter_host = driver.find_element_by_id('host-of-table-with-filter')

driver.switchToSubTree(table_with_filter_host)

dropdown_filter_host = driver.find_element_by_id('host-of-dropdown-filter')

driver.switchToSubTree(dropdown_filter_host)

select = driver.find_element_by_tag_name('select')

select.click()

select.find_element_by_xpath('//[text()=="Some Filter"]').click()

driver.switch_to_default_content()

driver.switchToSubTree(table_with_filter_host)

table_host = driver.find_element_by_id('host-of-table')

driver.switchToSubTree(table_host)

rows = driver.find_elements_by_xpath("//tr")

assert(len(rows), 5)

In this simple example, 8 of 13 lines of code are just navigating the shadow DOM.  Navigating the shadow DOM is over 60% of the test but provides no benefit.

Concern #3: Unlike an iframe, there could easily be tens to hundreds of shadow DOMs in a document, and they may be nested deeply.

I have never ran into nested iframes.  I am having nightmares about testing my teams application, which has shadow DOM going at least 5 layers deep in some places.

Concern #4: It is impossible to switch between shadow DOMs without first reseting to the default content, and drilling down one shadow DOM at a time.

Take the example in Concern #2 and lets say in the same test I want to test another filter, I would have to add.

driver.switch_to_default_content()

driver.switchToSubTree(table_with_filter_host)

driver.switchToSubTree(dropdown_filter_host)

select.click()

select.find_element_by_xpath('//[text()=="Another Filter"]').click()

driver.switch_to_default_content()

driver.switchToSubTree(table_with_filter_host)

driver.switchToSubTree(table_host)

rows = driver.find_elements_by_xpath("//tr")

assert(len(rows), 0)

I gain almost nothing from having previously located the shadow hosts and elements I care about.

Possible solution #1 (from me):  Allow interaction with ANY element, as long as the element.ownerDocument == current frame.  Rename switchToSubTree() to something that indicates you are only changing the search context of driver.find_element and driver.find_elements, like switchSearchContextToSubTree().

Pros:

Once you located the element(s) you want to interact with, you don't have to worry about calling switchSearchContextToSubTree() anymore.  You could call element.text or element.click() no matter what your search context was.

You could use javascript to return elements in nested shadow DOM, circumventing the need for multiple switchSearchContextToSubTree() calls in the first place.

Cons:

Blurs the lines of shadow DOM encapsulation.  However I would argue that shadow DOM encapsulation is an implementation detail that makes life easier for a web developer.  It is not a security feature and is completely transparent to an actual user of a web page, so I don't see the harm in blurring the lines when simulating a user with webdriver.

Possible solution #2 (from simonstewart): Provide a method like driver.switchToOwningSubTree(<element>)

Pros:

Once you located the elements you want to interact with, interacting with them is one switchToOwningSubTree() call away.

You could jump between shadow DOMs.

Cons:

Still have to change contexts a lot, (but it is only 1 line of code instead of multiple lines at least).

These are just 2 solutions, and I am sure there are other ones.  The main point of this e-mail was to spark discussion about the current proposal for webdriver + shadow DOM and some possible drawbacks of it.

[Ken Kania]

I don't have any experience using shadow DOM for real apps, but I'd still like to draw attention to a 3rd alternative:

Add getShadowRoot(int index) to WebElement, which will return a reference to the shadow root hosted by the element.  Unlike following the switch to model, elements from any shadow root (or the document) could be intermixed in calls to execute script, and it would be possible to click/type/etc an element without switching to the appropriate DOM tree first.

Note, I'm not proposing that findElement should be able to return elements outside the current DOM, but just that you could interact with an element without having to switch between trees.  This is similar to the Zachary's first option, except I don't think it's worthwhile for the user to just change search contexts on the driver object.

Ken

Ken

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-develo...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jason Leyba]

I like your proposal Ken, but why would WebElement#getShadowRoot() need an index?  According to the shadow DOM spec, only the youngest shadow root is accessible from a DOM element.

Approaching this from the end user API, I think we should have something like:

interface WebElement {

  /**

   * Returns a search context that represents this element's current shadowRoot.

   * This context may be used to find elements under the shadowRoot.

   *

   * @throws NoSuchElementException if this element is not a shadow host.

   */

  ShadowRoot getShadowRoot();

}

/**

 * Represents the root of a shadow tree.

 */

interface ShadowRoot extends SearchContext {

  /**

   * Locates an element under this root in the shadow DOM.

   * @throws StaleElementReferenceException If this ShadowRoot is no longer

   *

   * @throws NoSuchElementException if the element cannot be found.

   */

  @Override

  WebElement findElement(By locator);

  /**

   * Locates multiple elements under this root in the shadow DOM.

   * @throws StaleElementReferenceException If this root is no longer attached

   *     to its host.

   */

  @Override

  List<WebElement> findElements(By locator);

  /**

   * Returns the element currently focused in the shadow tree.

   * @throws NoSuchElementException If no element's in this root have focus.

   */

  WebElement getActiveElement();

}

Sample usage based on Zachary's first example:

tableWithFilterHost = driver.findElement(By.id("host-of-table-with-filter"));

select = tableWithFilterHost

    .getShadowRoot()

    .findElement(By.id("host-of-dropdown-filter"))

    .getShadowRoot()

    .findElement(By.tagName("select"));

select.click()

select.findElement(By.xpath("//[text()==\"Some Filter\"]").click();

rows = tableWithFilterHost

    .getShadowRoot()

    .findElement(By.id("host-of-table"))

    .getShadowRoot()

    .findElements(By.xpath("//tr"));

assertEquals(5, rows.size());

A WebElement may be used as long as WebDriver is focused on its ownerDocument - same semantics as frames.

As for the wire protocol, it'd be up to each driver implementation to encode a WebElement so it could be found in the current document.  This would depend on driver implementation details, but you could have something like:

{"ELEMENT": "one"}            // Belongs to the main document tree

{"ELEMENT": "one:two"}        // In the shadow tree of {"ELEMENT": "one"}

{"ELEMENT": "one:two:three"}  // In the shadow tree of {"ELEMENT": "one:two"}

-- Jason

[Jason Leyba]

On Sat, Aug 17, 2013 at 8:58 AM, Jason Leyba <jml...@gmail.com> wrote:

I like your proposal Ken, but why would WebElement#getShadowRoot() need an index?  According to the shadow DOM spec, only the youngest shadow root is accessible from a DOM element.

 Sorry, misread the spec - we would need the index.  That doesn't change anything else though.

[Simon Stewart]

Finally catching up on posts that I need to reply to. 

Is anyone from Google planning on raising this at the F2F in China? I need some time to reacquaint myself with the Shadow DOM before I reply to this thread with confidence. My main concerns with the APIs are:

1) How would a developer familiar with the code jump to the source file that generated this part of the DOM? The switchTo statements are verbose, but leave breadcrumbs for this.

2) How would an inexperienced tester, asked to write code for the first time, figure out how to get hold of the right part of the Shadow DOM? We already know that just looking at the source of the page won't give a clue, and last I looked the Web Inspector in Chrome didn't make this a simple operation. That was a while ago, so it'd be nice if that had changed. Again, the model of the current API makes this a tedious but possible task.

3) Can this be implemented in the browser without breaking their internal models? The render tree is quite different from the logical DOM, and doesn't tend to be accessible, so (as a straw man argument) any suggestion that relies on exposing that is going to be an uphill struggle to get implemented.

4) How often must someone touch this? The Advanced User Interaction APIs are pretty nasty to work with at a low level, but we've provided encapsulations which make it a more palatable experience. 

Cheers,

Simon 

--

[Marc Fisher]

On Sunday, November 3, 2013 2:44:24 PM UTC-8, Simon Stewart wrote:

Finally catching up on posts that I need to reply to. 

I will be at the F2F in China and want to discuss this.

In other discussions we have already moved beyond the original idea proposed by Zach and have submitted a proposed change based on Ken's/Jason's suggestion in this thread. We have written up this proposal and attached it to a bug in bugzilla (see https://www.w3.org/Bugs/Public/show_bug.cgi?id=22987).

The main point that answers most of your concerns below is that the initial interaction to get to the shadow dom is basically the same. The developer locates the node with a shadow root attached, then performs an operation on that node to get to that shadow root. The difference is that the original proposal changed the global context to that root, affecting all operations thereafter until another switchto is performed, and the new proposal instead returns a WebElement that corresponds to that root, allowing operations within that dom to be freely interleaved with operations that represent other portions of the page.

Marc

[Shuotao Gao]

I've uploaded a new patch to bug https://www.w3.org/Bugs/Public/show_bug.cgi?id=22987.

@Simon, would you like to review it?

--

You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-develo...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/539f16d5-dd82-4d07-963e-986198a8aaf7%40googlegroups.com.