Intent to Deprecate and Remove : URL.createObjectURL with MediaStream

[Florent Castelli‏]

Contact emails

orp...@chromium.org, gui...@chromium.org

Spec

https://w3c.github.io/mediacapture-main/

Summary

This functionality has been removed from the spec in 2013 and superseded by assigning streams to HTMLMediaElement.srcObject directly.

Motivation

Other UAs have already deprecated (Firefox) or removed (Safari) the feature and we should align our implementation with theirs.

The old way is less safe as you need to remember to call URL.revokeObjectURL manually. The new API is object based and will make use of the GC properly.

Interoperability and Compatibility Risk

The interoperability risk should be small as other UAs already deprecated or removed the functionality. Also, all UAs currently support the new API as intended by the spec.

The compatibility risk is low as the feature is not used much (see below).

Firefox: Supported (deprecation shipped in 54, removal soon).

Edge: No public signals

Safari: Supported (feature already removed).

Web developers: No signals

Alternative implementation suggestion for web developers

Assigning the streams directly to HTMLMediaElement.srcObject will have the same effect.

Usage information from UseCounter

Considering Safari already removed the feature, I would expect more sites to migrate to the newer API.

Currently, HTMLMediaElement.srcObject is used ~6 times more often than URL.createObjectURL for MediaStreams (0.039% vs. 0.0064%)

Metrics for URL.createObjectURL(mediastream): https://www.chromestatus.com/metrics/feature/timeline/popularity/1606

Metrics for HTMLMediaElement.srcObject: https://www.chromestatus.com/metrics/feature/timeline/popularity/1603

OWP launch tracking bug

https://crbug.com/800767

Entry on the feature dashboard

https://www.chromestatus.com/features/5618491470118912

Requesting approval to remove too?

Yes. We plan to deprecate in M66 and remove in M68.

[Philipp Hancke‏]

note that there is a use-case which isn't covered by srcObject described here:

    https://github.com/webrtc/adapter/issues/636

As Martin (mozilla) points out it probably shouldn't work today and it seems to be used quite rarely.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHQbFacoPd2Z33PQ4Rf9DJU0kZZeBa9Q8dzDtWvM1ARTCG7qZw%40mail.gmail.com.

[Philip Jägenstedt‏]

This is a tricky one.

The usage is still pretty high, ~0.01% right now. Once upon a time URL.createObjectURL was the only way to attach a MediaStream to a media element, so there's probably some old-ish code that isn't being updated much that will break.

I commented on https://github.com/webrtc/adapter/issues/636 to ask if this was being done on a "real" site. API shape aside, it's a pretty reasonable thing for a top-level frame to have a MediaStream and delegating that to a cross-origin iframe. (Note that just moving the conversation to a separate window doesn't need this feature, though, one could create a same-origin window and just assign a mediaElement.srcObject within, modulo bugs.)

And still, Safari 11 indeed doesn't support this, and the suggestion to reinstate URL.createObjectURL(mediaStream) in the spec didn't go anywhere, so at this point actually following through with the removal is probably the best way to interop.

I think we should do this, but with some extra caution. Florent, do you think that 4 release cycles of deprecation would be acceptable here, or is the timing important? While it's deprecated, another thing we could do is to disable it if experimental web platform features are enabled, to get some error reports from people using Canary. If we're lucky, the 0.01% might be just a few large-ish web properties, and the problem might go away!

WDYT?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHQbFacoPd2Z33PQ4Rf9DJU0kZZeBa9Q8dzDtWvM1ARTCG7qZw%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKi%2BbTYusiv73BAcmXk9o%3DWaLN4K80htvxZNPNhaE%3Dvzvbw%40mail.gmail.com.

[Marijn Kruisselbrink‏]

On Tue, Jan 16, 2018 at 6:28 AM, 'Philip Jägenstedt' via blink-dev <blin...@chromium.org> wrote:

This is a tricky one.

The usage is still pretty high, ~0.01% right now. Once upon a time URL.createObjectURL was the only way to attach a MediaStream to a media element, so there's probably some old-ish code that isn't being updated much that will break.

I commented on https://github.com/webrtc/adapter/issues/636 to ask if this was being done on a "real" site. API shape aside, it's a pretty reasonable thing for a top-level frame to have a MediaStream and delegating that to a cross-origin iframe. (Note that just moving the conversation to a separate window doesn't need this feature, though, one could create a same-origin window and just assign a mediaElement.srcObject within, modulo bugs.)

Does the delegating to a cross-origin iframe actually work in chrome today? MediaStreamRegistry::RegisterURL only has a url->stream mapping within the one renderer process, so at least out-of-process-iframes would break that use case anyway afaict.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHQbFacoPd2Z33PQ4Rf9DJU0kZZeBa9Q8dzDtWvM1ARTCG7qZw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKi%2BbTYusiv73BAcmXk9o%3DWaLN4K80htvxZNPNhaE%3Dvzvbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYfn-JRVM3itYaNZnWz%2BBKw_hfvM%3DPDhxFMMy4e%2B2qPHWw%40mail.gmail.com.

[Philip Jägenstedt‏]

On Tue, Jan 16, 2018 at 6:56 PM Marijn Kruisselbrink <m...@chromium.org> wrote:

On Tue, Jan 16, 2018 at 6:28 AM, 'Philip Jägenstedt' via blink-dev <blin...@chromium.org> wrote:

This is a tricky one.

The usage is still pretty high, ~0.01% right now. Once upon a time URL.createObjectURL was the only way to attach a MediaStream to a media element, so there's probably some old-ish code that isn't being updated much that will break.

I commented on https://github.com/webrtc/adapter/issues/636 to ask if this was being done on a "real" site. API shape aside, it's a pretty reasonable thing for a top-level frame to have a MediaStream and delegating that to a cross-origin iframe. (Note that just moving the conversation to a separate window doesn't need this feature, though, one could create a same-origin window and just assign a mediaElement.srcObject within, modulo bugs.)

Does the delegating to a cross-origin iframe actually work in chrome today? MediaStreamRegistry::RegisterURL only has a url->stream mapping within the one renderer process, so at least out-of-process-iframes would break that use case anyway afaict.

I tried to make something that works just now, but couldn't.

I had this in one origin (Live DOM Viewer)

<!DOCTYPE html>

<button>do it</button><br>

<iframe src="https://other.origin/bloburl.html"></iframe>

<script>

var button = document.querySelector('button');

button.onclick = function() {

  navigator.mediaDevices.getUserMedia({video: true}).then(ms => {

    var url = URL.createObjectURL(ms);

    var iframe =document.querySelector('iframe');

    iframe.contentWindow.postMessage(url, '*');

  });

}

</script>

And this was the bloburl.html in the other origin:

<!doctype html>

<video controls></video>

<script>

addEventListener('message', event => {

  var url = event.data;

  document.querySelector('video').src = url;

});

</script>

And that failed with this message in the console: "Not allowed to load local resource: blob:https://software.hixie.ch/88c4bd1b-cc1e-4026-baef-a5ea0f95d70d"

Maybe I did something wrong, but hopefully then this never worked. And yes, if it had, then OOPIF would break it.

I didn't read the original code snippet in that issue very carefully, I commented again about how it was apparently a same-origin case all along.

It might still be a bit of a nuisance to rewrite code that's now passing URLs between same-origin windows to pass objects instead. But I'd put that roughly in the same complexity ballpark as switching from .src to .srcObject in the first place, where there will no doubt be some code where the media stream creation and media element are separated by layers of code that only talk URLs.

[Harald Alvestrand‏]

Yeah, the implementation of srcObject was delayed 6-12 months while we asked for others to tell us which way the dependencies should go. The code was the easier part.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdm-M9VjYdx2Lk9EusKehXem%2B%3DJeyeBZ6xkrBKPg%2BdGmw%40mail.gmail.com.

[Philip Jägenstedt‏]

Well, that was also no fun :) Here I was thinking about web developers' code, where perhaps they're using some library for media controls that want a URL, and some work will be needed to get a MediaStream object down onto the media element's srcObject.

[Florent Castelli‏]

I reused your example and tried it on a same-origin iframe and it worked. I could also just pass the stream directly to the iframe and it worked too.

I believe that OOPIF will certainly prevent that use-case from working, so this shouldn't be a blocker for deprecation.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Philip Jägenstedt‏]

Thanks for checking that, Florent, good to confirm that passing MediaStream objects to other windows works, which it should, but bugs wouldn't have been surprising.

So I think we should do this. About the deprecation period, we've discussed off-list. Let's go with the suggest 2 release cycles, but see if the usage has moved at all before the M58 branch point. Usage normally doesn't seen to be affected by deprecation messages, but we'll see.

LGTM1

On Wed, Jan 17, 2018 at 2:25 PM Florent Castelli <orp...@google.com> wrote:

I reused your example and tried it on a same-origin iframe and it worked. I could also just pass the stream directly to the iframe and it worked too.

I believe that OOPIF will certainly prevent that use-case from working, so this shouldn't be a blocker for deprecation.

[Florent Castelli‏]

Indeed, if the usage of this API does not go down before the M68 branch point, we can extend the deprecation period accordingly and give more time for users to migrate their code.

LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Mike West‏]

LGTM2. The numbers look low enough for deprecation to be successful. Give it a shot!

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHQbFadrhDDSfk4ghonBt%3DHM8YJ9X6h1v4c5ErNESXVkAL6mKg%40mail.gmail.com.

[Rick Byers‏]

LGTM3 - good luck!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3Df-Ump6eJV%2B58sp-Gn9U-arRK-GfS6s779Jw39E4rS7Qg%40mail.gmail.com.

[Sam Dutton‏]

FWIW I've done a PR to update the HTML5 Rocks WebRTC articles (which still get a lot of views).