Intent to Implement: HTMLMediaElement.srcObject for MediaStreams

[Guido Urdaneta]

Contact emails

gui...@chromium.org, h...@chromium.org

Spec

http://w3c.github.io/mediacapture-main/#mediastreams-in-media-elements

https://html.spec.whatwg.org/multipage/embedded-content.html#media-elements

Tag review:https://github.com/w3ctag/spec-reviews/issues/57  

Summary

The HTML spec specifies that HTMLMediaElements have the srcObject attribute, which makes it possible to associate a MediaProvider with the media element. A MediaProvider can be a MediaStream, MediaSource or Blob.

About 18 months ago, there was an attempt to add support for the srcObject attribute to Blink. However, the attempt failed due to difficulties related to Blink’s dependency rules. Basically, given that HTMLMediaElement and Blob are part of core, and MediaStream and MediaSource are in separate modules, it was difficult to produce an implementation that was simple and clean enough to pass code review without violating any dependency rules.

Since then, all other major browsers have added support for the srcObject attribute, but they provide support only for MediaStreams. Doing the same in Blink without violating any dependency rules would be straightforward. We simply have to define the srcObject attribute in a partial interface of HTMLMediaElement in the mediastream module, and make some changes in HTMLMediaElement without requiring HTMLMediaElement to know about MediaStreams.

This intent covers the addition of an srcObject attribute of type MediaStream to HTMLMediaElement. We acknowledge that it would be desirable to implement a general solution that also provides support for Blob and MediaSource, but, to the best of our knowledge, the dependency-related difficulties have not been solved yet. We believe it is better to initially add support for MediaStreams, improving interoperability with the other browsers, and defer the general solution until projects such as Onion Soup make it possible to implement such a solution in a more straightforward manner.

Motivation

All major browsers support or plan to support an srcObject attribute of type MediaStream.

While this is not a complete implementation of the spec, we believe everyone would benefit from having at least the same level of support in Blink as in the other browsers.

Some relevant links:

Edge:

https://blogs.windows.com/msedgedev/2015/05/13/announcing-media-capture-functionality-in-microsoft-edge/

https://dev.windows.com/pt-pt/microsoft-edge/platform/documentation/apireference/interfaces/htmlmediaelement/

Firefox:

https://www.fxsitecompat.com/en-CA/docs/2015/htmlmediaelement-srcobject-has-been-unprefixed/

https://hg.mozilla.org/mozilla-central/file/584870f1cbc5d060a57e147ce249f736956e2b62/dom/webidl/HTMLMediaElement.webidl

WebKit:

https://bugs.webkit.org/show_bug.cgi?id=121943

https://github.com/WebKit/webkit/blob/master/Source/WebCore/Modules/mediastream/HTMLMediaElementMediaStream.idl

http://www.webrtcinwebkit.org/blog/

Interoperability and Compatibility Risk

The addition of the MediaStream-only srcObject attribute improves interoperability, as it makes Blink more compliant with the spec and more compatible with WebKit, Edge and Firefox, which already support the attribute. The lack of support for Blob and MediaSource does not introduce interoperability problems, as the other browsers do not support them either and have not shown explicit signals in favor of supporting those types in the near future.

We do not anticipate any compatibility risk, as the existing mechanism to support similar functionality (URL.createObjectURL() together with the src attribute) will continue to be supported, so existing applications do not need to be rewritten.

With regards to a future refactoring in Blink to add support for MediaSource and Blob, we do not anticipate such refactoring to require significantly more effort than implementing a complete solution from scratch without previous support for MediaStreams.

Ongoing technical constraints

None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=595326

Link to entry on the feature dashboard

https://www.chromestatus.com/feature/5989005896187904

Requesting approval to ship?

No.

[Rick Byers]

This sounds like a great middle ground to get most of the value without having to solve the hard implementation problems.  Thanks!

I'd be curious to know why all the other vendors have chosen to support only this subset.  If no vendor has concrete plans to support MediaSource or Blob then perhaps the spec should just be changed to match implementation reality?

[Guido Urdaneta]

I looked a bit further and found this Firefox thread from July 2015 with the intent to unprefix srcObject with support for MediaStream only.

They state that they still want to support the other types eventually.

https://groups.google.com/forum/m/#!topic/mozilla.dev.platform/wBV5TNDEziU

In 2014 there was a patch to add support for MediaSource, but it was reverted and has not been relanded since. (https://bugzilla.mozilla.org/show_bug.cgi?id=886194)

I still think it's a good plan to first add srcObject support for MediaStream, and add support the other types when it becomes easier to do so.

[Rick Byers]

Sounds reasonable to me, thanks!

[Philip Jägenstedt]

I've been following the srcObject situation since the ill-fated attempt 18 months ago. The discussion has been spread out in many places, and here's what I said in a private thread in the last round:

An option which I think we never discussion would be a separate modules/srcobject where the union type would be allowed, and one could have a partial interface HTMLMediaElement with srcObject as the only attribute. That wouldn't require any custom bindings, and the implementation could simply wrap the type in a platform/ type called MediaProviderWrapper or some such that could be given to the real HTMLMediaElement without violating any layer rules, that could later be used to create the WebMediaPlayer, also outside of core/.

I still think this is doable. Given that other browsers have already shipped with support for only MediaStream, I think doing the same is the right choice, but it would be good if it's done as a separate srcobject module so that adding MediaSource and Blob support is just a matter of extended the solution to cover more cases. If it should turn out during code review that this is actually significantly harder than a MediaStream-only solution then it'll have to wait, however.

What timeline do you have in mind for shipping?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Guido Urdaneta]

I'll try the modules/srcobject approach. Didn't know it was already possible to make a module depend on other modules.

With regards to the timeline, the plan is to have it implemented this week. If we all think the implementation is good, then ship before M51 branches.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Kentaro Hara]

The modules/srcobject approach sounds reasonable to me. (Sorry that the core - modules componentization makes the implementation of HTMLMediaElement.srcObject complicated.)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--

Kentaro Hara, Tokyo, Japan