WebDriver video/audio element interface

[Andriy Prystupa]

Hi All,

I have some topic to discuss about video/audio element testing by WD.
Current implementation allows to control player element only with javascript, ie play/pause can be issued only by executing JS.

As I understand HTML5 video/audio element implementation is browser specific. And player's UI is not accessible by WD.
What do you think about to add specific player API to WebDriver specification?
In example:

• /session/:sessionId/element/:id/player/speed
• /session/:sessionId/element/:id/player/state
• /session/:sessionId/element/:id/player/volume
• /session/:sessionId/element/:id/player/seek
• /session/:sessionId/element/:id/player/mute

If element is some media then it supports commands listed above. This will simplify UI with media testing on Selenium side. And this will allow to implement WD interface for testing video elements on mobile UIs(Appium, Selendroid, etc) as well I guess.

We checked this idea in own fork of WD/Selenium and it works.

Regards,
Andriy.

[Seva Lo]

Hi Andriy,

Let's see. HTML spec defines a bunch of attributes and verbs for the media elements. http://www.w3.org/html/wg/drafts/html/master/#media-elements

To get values of attributes (e.g. "volume"), the existing GET /session/:sessionId/element/:id/attribute/:name endpoint will do.

There is no POST for /session/:sessionId/element/:id/attribute/:name and there's no WebElement.setAttrbute() method in the bindings. SetAttribute makes sense to me at the first glance but I suspect there was a discussion about it in the past... Was there a discussion/decision about WebElement.setAttribute()?

Next, HTML spec defines these methods/verbs on the media elements: load(), canPlayType(), fastSeek(), getStartDate(), play(), pause(), addTextTrack(). Should we propagate all of them to WebDriver API? None of them? Ones most useful for web testing? Whatever we decide to propagate, I'd be for keeping spelling, e.g. use "fastSeek", not "seek".

If we decide to propagate some of the media elements' verbs to WebDriver wire protocol, we'd need to find a place to put them in the client bindings:

1. Add the methods (e.g. load(), start(), pause()) directly to WebElement interface/API and throw error every time anyone calls them on a non-media element?..

2. Proactively determine type of every element at runtime and add these methods (or +augment an interface - depending on the language) to every WebElement which is a media element?

3. Introduce some Media interface with those methods and make user call element.media().pause() and again fail that if it's not a media element?

I am not sure I like any of these options. Do you have a suggestion here, Andriy? Anyone else?

Of course, if any new wire protocol endpoints will be added, we will need a capability (because not all browsers will support media elements and not all drivers will provide the API), though that is relatively straightforward.

-- 

Seva

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-develo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-developers/fe2d6ce8-b712-49b6-b47a-082743e164ac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.