Intent to Deprecate and Remove: 'stalled' events for HTMLMediaElements using MediaSource

Visto 503 veces
Saltar al primer mensaje no leído

Chrome Cunningham

no leída,
23 abr 2018, 16:23:2223/4/18
a blink-dev,Mounir Lamouri,Dale Curtis,Matthew Wolenetz

Primary eng

chcunn...@chromium.org


Summary

The 'stalled' event is raised if the media download has failed to progress for ~3 seconds. When using MSE, the web app manages the download (xhr) and the media element is not aware of its progress.


Web authors using MSE have noticed that 'stalled' seems to be emitted at random times.

https://bugs.chromium.org/p/chromium/issues/detail?id=517240


Due to code cruft, Chrome will raise 'stalled' whenever app has not appended new media (SourceBuffer.Append(...)) in the last 3 seconds. Apps may choose to append large chunks of data at a much lower frequency, so this is not a useful signal about buffering health.


Removing the 'stalled' for MSE should clear up confusion and bring us more in line with the spec (https://github.com/w3c/media-source/issues/88#issuecomment-374406928).


Media elements that don't use MSE will continue to raise 'stalled' as they do today.


Interoperability and Compatibility Risk

All browsers bellow appear to have the same behavior as current Chrome.


Edge: Haven't heard back yet

Firefox: Supportive (https://bugzilla.mozilla.org/show_bug.cgi?id=1451149

Safari: Supportive (email)


Alternative implementation suggestion for web developers

MSE users typically fetch the media chunks via XHR. XHR provides a number of events for monitoring the progress of the download.


https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Using_XMLHttpRequest#Monitoring_progress


Usage information

We don't have a good use-counter for this. Folks probably are listening for the event (e.g. unknowingly as part of a video streaming library). But listening != meaningful usage.


I polled the video streaming community at video-dev.slack.com (developers representing many major streaming sites/platforms). None were using 'stalled'.


One nuance is the opposite event: 'progress', which is not being deprecated. For non-MSE media elements, 'progress' indicates the download is moving along. For MSE, 'progress' has come to mean that the range of what's buffered has changed. This meaning is not in the spec, but has been baked into Chrome's implementation since the MSE shipped and other UAs seem to do the same. In a separate video-dev poll, many web authors using MSE indicated they listen for 'progress' as a trigger to update the part of their custom UI that shows how much of the stream is buffered. There are alternative means of achieving this, but breaking these folks is not worth the cost - 'progress' can stay.


Entry on the feature dashboard

None. This is a very small removal. Happy to create one if desired.


Requesting approval to remove too?

Yes, in M68.


Anne van Kesteren

no leída,
24 abr 2018, 3:27:3224/4/18
a Chrome Cunningham,blink-dev,Mounir Lamouri,Dale Curtis,Matthew Wolenetz
On Mon, Apr 23, 2018 at 10:23 PM, Chrome Cunningham
<chcunn...@chromium.org> wrote:
> The 'stalled' event is raised if the media download has failed to progress
> for ~3 seconds. When using MSE, the web app manages the download (xhr) and
> the media element is not aware of its progress.

I checked with the HTML Standard and it seems it's only supposed to
fire for remote resources. Will you also be removing it for File and
MediaStream objects?

Are there web-platform-tests?


--
https://annevankesteren.nl/

Joe Medley

no leída,
24 abr 2018, 12:31:4324/4/18
a chcunn...@chromium.org,blink-dev,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org
Can you please create a status entry for this? 
Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195
If an API's not documented it doesn't exist.


--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c9c57bdc-e3f3-4cb7-bb34-ea01b58b64fb%40chromium.org.

Mike West

no leída,
25 abr 2018, 4:21:4725/4/18
a chcunn...@chromium.org,blink-dev,Mounir Lamouri,dalec...@chromium.org,wole...@chromium.org
Removing without warning seems risky in the absence of any data about usage. Do we have a feel for the origins where MSE sees large-scale usage? Have we verified that the lack of this event won't break those sites?

It also seems a little strange that we'd distinguish between MSE content and non-MSE content for this event. If it isn't useful (or used) for MSE content, it might be possible to remove the event entirely rather than bifurcating our behavior based on (seemingly?) unrelated qualities of the content being rendered. Did y'all consider that possibility?

-mike

Chrome Cunningham

no leída,
25 abr 2018, 17:39:3225/4/18
a blink-dev,chcunn...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,Miguel Casas-Sanchez,Marijn Kruisselbrink,Steven Robertson
+Marijn for "File"
+Miguel for "MediaStream"

> I checked with the HTML Standard and it seems it's only supposed to 
fire for remote resources. Will you also be removing it for File and 
MediaStream objects? 

I agree with your take on the standard. I had planned to only change MediaSource. This deprecation is a low priority technical debt cleanup - IMO, its only worth pursuing because I'm confident that the risk is very small (demonstrated by wide community survey and my inability to create even a hypothetical breakage). I'm not familiar enough to say whether that's true for the other sources.

> Are there web-platform-tests?

There are no tests that expect this event to be fired for MSE. There is one test that explicitly expects it is NOT fired. The ideal post-deprecation test would verify that the event is NEVER fired. This is a bit difficult ;). We could write a test that waited for 3 seconds (ugh) between sourceBuffer appends to ensure that at least the current firing behavior has gone away. Not sure if that's worth doing.

> Can you please create a status entry for this? 


> Do we have a feel for the origins where MSE sees large-scale usage? Have we verified that the lack of this event won't break those sites?

We do have this data. I have not systematically verified for the top N sites, but the poll I performed in the video-dev community gave us really good coverage. In no particular order, we heard "won't break" from devs representing:
Netflix, Vimeo, Brighcove, Ingest.io, Soundcloud, Jwplayer, RealEyes, NyTimes.com, Bitmovin, DailyMotion

+strobe@ for YouTube (haven't asked yet)

> Removing without warning

We could add a console warning (e.g. for any MSE <video> element with 'stalled' event listeners) if this would address your concern? IMO this is not needed.

> If it isn't useful (or used) for MSE content, it might be possible to remove the event entirely rather than bifurcating our behavior based on (seemingly?) unrelated qualities of the content being rendered. Did y'all consider that possibility?

MSE vs non-MSE (or in spec-speak: 'local' vs 'remote' resources) is a meaningful distinction and bifurcating in this way is what the spec would have us do. The 'stalled' event has solid meaning/use-case for non-MSE playbacks where the UA manages the download. No desire deprecate for those.  

Chris

Marijn Kruisselbrink

no leída,
25 abr 2018, 17:50:2725/4/18
a Chrome Cunningham,blink-dev,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,Miguel Casas-Sanchez,Steven Robertson


On Wed, Apr 25, 2018 at 2:39 PM, Chrome Cunningham <chcunn...@chromium.org> wrote:
+Marijn for "File"
Not sure what the question is? Following the spec better sounds good to me, but I'm not familiar with the media side of things. So I guess that would mean not firing "stalled" events for any srcObject or valid blob: URL?

Anne van Kesteren

no leída,
26 abr 2018, 3:25:3326/4/18
a Marijn Kruisselbrink,Chrome Cunningham,blink-dev,Mounir Lamouri (via Google Docs),Dale Curtis,Matt Wolenetz,Miguel Casas-Sanchez,Steven Robertson
On Wed, Apr 25, 2018 at 11:50 PM, Marijn Kruisselbrink <m...@chromium.org> wrote:
> Not sure what the question is? Following the spec better sounds good to me,
> but I'm not familiar with the media side of things. So I guess that would
> mean not firing "stalled" events for any srcObject or valid blob: URL?

Yeah, that's what the HTML Standard requires.


--
https://annevankesteren.nl/

Daniel Bratell

no leída,
26 abr 2018, 11:28:5326/4/18
a blink-dev,Chrome Cunningham,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,Miguel Casas-Sanchez,Marijn Kruisselbrink,Steven Robertson
I support this change, but just a question/request. Is it possible to find stalled event handlers for MSE media in a code search (at github, in Google's archive if you have access to that or somewhere else) to get an idea if they even exist?

Good that you added a status entry for this. Even if it's a small change, this way it will be listed in platform changelogs.

Not sure a deprecation phase is needed. Maybe a code search will tell us.

/Daniel
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5a950f89-b6c0-45d6-a313-f47aa570fe70%40chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

Chrome Cunningham

no leída,
26 abr 2018, 19:30:1226/4/18
a blink-dev,chcunn...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,m...@chromium.org,str...@chromium.org
Miguel, friendly ping for thoughts on MediaStream.

Marijn: Are you of aware of any use case for 'stalled' with File? Any objection to not firing it as the spec suggests? 
I can help from the media side. Do you have any examples of media playbacks using srcObject=File? It may be that this event is already not fired in this case. 

> +strobe@ for YouTube (haven't asked yet)
Strobe replied back, but he isn't part of this group. YouTube is not using 'stalled' with MSE. 

> Is it possible to find stalled event handlers for MSE media in a code search (at github, in Google's archive if you have access to that or somewhere else) to get an idea if they even exist?

I did not find any uses of 'stalled' with MSE in google's codebase. Its not trivial to search all of github... need to narrow it to an owner, or repo. 

I suspect it *is* possible to find people who are listening,  but this is a red herring. In the examples I have where folks listen to this event with MSE, they expected the event indicated poor video buffering health and observing the event triggered various metrics reporting. This usage is simply wrong. Eliminating that confusion is the main goal of this cleanup. 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Harald Alvestrand

no leída,
27 abr 2018, 0:45:2427/4/18
a Chrome Cunningham,blink-dev,mlam...@chromium.org,Dale Curtis,Matthew Wolenetz,Miguel Casas-Sanchez,Marijn Kruisselbrink,str...@chromium.org
httparchive contains 13 mentions of "onstalled" together with "srcObject".

I think this is small enough to be able to say that "there's no significant usage of onstalled together with MediaStreams".

The most common pattern in this set of 13 was a line saying:

declare var onstalled: (this: Window, ev: Event) => any

which is not comprehensible to me, but "feels" like setting up handlers for "anything else that happens".

Miguel Casas

no leída,
27 abr 2018, 2:26:1327/4/18
a Chrome Cunningham,gui...@chromium.org,blink-dev,Mounir Lamouri,Dale Curtis,Matt Wolenetz,m...@chromium.org,str...@chromium.org
On 26 April 2018 at 19:30, Chrome Cunningham <chcunn...@chromium.org> wrote:
Miguel, friendly ping for thoughts on MediaStream.


 I have no strong opinions about it: from the POV of capture-from-element, this event is irrelevant; +guidou@ for a <video> with a MediaStream as feeder.



--

Miguel Casas-Sanchez | Gatopardo del Software | ydog / mca...@google.com | +1 (650) 603 1380

Steven Robertson

no leída,
27 abr 2018, 2:27:4927/4/18
a Chris Cunningham,blin...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,m...@chromium.org,str...@chromium.org
yeah we don't even listen to it in YT

Marijn Kruisselbrink

no leída,
27 abr 2018, 13:34:3727/4/18
a Chrome Cunningham,blink-dev,mlam...@chromium.org,Dale Curtis,wole...@chromium.org,Miguel Casas-Sanchez,Steven Robertson
On Thu, Apr 26, 2018 at 4:30 PM, Chrome Cunningham <chcunn...@chromium.org> wrote:
Miguel, friendly ping for thoughts on MediaStream.

Marijn: Are you of aware of any use case for 'stalled' with File? Any objection to not firing it as the spec suggests? 
I can help from the media side. Do you have any examples of media playbacks using srcObject=File? It may be that this event is already not fired in this case. 

Sorry, no idea. Not firing as the spec suggests sounds good to me. In practice I expect it would indeed never be fired for blobs since unless something really funky is going on reading from memory and/or a file on disk should never stall for multiple seconds anyway...

Chris Harrelson

no leída,
30 abr 2018, 16:46:1230/4/18
a Marijn Kruisselbrink,chcunn...@chromium.org,blink-dev,Mounir Lamouri,Dale Curtis,Matt Wolenetz,Miguel Casas-Sanchez,str...@chromium.org
LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVZ9VhsHWw3HtT9qzZeH3J3G8PJCFDGySnPFyK5Tq6OiTw%40mail.gmail.com.

Chrome Cunningham

no leída,
1 may 2018, 19:00:361/5/18
a blink-dev,m...@chromium.org,chcunn...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,str...@chromium.org,chri...@chromium.org
> I checked with the HTML Standard and it seems it's only supposed to 
fire for remote resources. Will you also be removing it for File and 
MediaStream objects? 

Good news - neither File nor MediaStream objects currently fire stalled event.

In the File case, the HTMLMediaElement.networkState transitions to "idle" almost immediately, which cancels the associated timers. 

In the MediaStream case, the underlying media player (WebMediaPlayerMS) hardcodes the DidLoadingProgress() to return true, so the media element will never 
consider loading to have "stalled". 

On Monday, April 30, 2018 at 1:46:12 PM UTC-7, Chris Harrelson wrote:
LGTM1

Woot! Awaiting 2 more. 
LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
...

Daniel Bratell

no leída,
2 may 2018, 5:16:542/5/18
a blink-dev,Chrome Cunningham,m...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,str...@chromium.org,chri...@chromium.org
LGTM2

/Daniel
LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
...
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1d1d1eeb-2467-4112-a070-1bebd65f95b5%40chromium.org.

Yoav Weiss

no leída,
3 may 2018, 6:44:273/5/18
a Daniel Bratell,blink-dev,Chrome Cunningham,m...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,str...@chromium.org,chri...@chromium.org
LGTM3

Please add WPT tests that make sure this event never fires (timeout based tests are not great, but no tests is worse...)

Chrome Cunningham

no leída,
11 may 2018, 19:24:1211/5/18
a blink-dev,bra...@opera.com,chcunn...@chromium.org,m...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,str...@chromium.org,chri...@chromium.org
LGTM2

/Daniel
LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
...
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.



--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

chcunn...@google.com

no leída,
29 may 2018, 13:17:5929/5/18
a blink-dev,chcunn...@chromium.org,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org,mca...@chromium.org,m...@chromium.org,str...@chromium.org
FYI - this is landed now, but didn't quite make M68. Will go out with M69.

mohan...@gmail.com

no leída,
6 ago 2018, 14:24:066/8/18
a blink-dev,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org
Why is this autoplay feature not working??

gawa...@gmail.com

no leída,
15 mar 2019, 13:59:4515/3/19
a blink-dev,mlam...@chromium.org,dalec...@chromium.org,wole...@chromium.org
.

Responder a todos
Responder al autor
Reenviar
0 mensajes nuevos