Re: [blink-dev] Intent to ship: Media Capture from <audio> and <video>

[Miguel Casas-Sanchez]

Update (a bit overdue, but anyhow): the method captureStreamUntilEnded() was discussed in https://github.com/w3c/mediacapture-fromelement/issues/42 and subsequently removed.

On 30 June 2016 at 04:26, Philip Jägenstedt <foo...@chromium.org> wrote:

On Wed, Jun 29, 2016 at 6:31 PM Miguel Casas <mca...@google.com> wrote:

On 29 June 2016 at 04:46, Philip Jägenstedt <foo...@chromium.org> wrote:

Are the additions in https://codereview.chromium.org/2110573002/ intended to be covered by this intent?

Probably not, since I too see little value in baking into Blink something that can be done with a (trivial?) polyfill.

Spec bug?

I guess, but was anyone working on Blink involved in having it added to the spec, that we should talk to about it first?

 

In that review I have some doubts that captureStreamUntilEnded() is actually a useful addition to the web platform. It seems to me that the benefit of an API like this could be in guaranteeing that not a single frame or audio sample after some condition appears in the MediaStream, which cannot be done by reacting to a event,

What do you mean? I see this as: listen for onended from the <video>/<audio> and finish the MediaStream when and if.

I mean that it's possible for scripts to do things after mediaElement.ended becomes true but before the ended event has fired. For example if one seeks it's not obvious if this might result in some video or audio frames from the new position passing through the MediaStream before it's closed. Maybe not, but that kind of guarantee is the main reason I could see for an API of this sort.

[Philip Jägenstedt]

There was already 1 LGTM from Chris, so LGTM2.

[Rick Byers]

[Thread is here in case anyone else needs a refresher on the history here]

Philip, I take it this means you're OK with the issues you filed not blocking shipping?

So Firefox ships a prefixed version of this API that has a number of "quirks", right?  Have we asked anyone there if they feel the spec'd behavior looks reasonable to them (i.e. mature enough that they'll probably be OK implementing it at some point)?

[Philip Jägenstedt]

Maybe I was a bit fast there. Mine aren't the only issues, there are a bunch that don't seem to be getting much attention:

https://github.com/w3c/mediacapture-fromelement/issues

Miguel, could you go through the open issues and comment what behavior you intend to ship? Where possible, adding to https://github.com/w3c/web-platform-tests/tree/master/mediacapture-streams to determine the behavior of Firefox would be helpful.

[Miguel Casas-Sanchez]

A bit of an update Spec- and implementation-wise:

- captureStreamUntilEnded() was removed from the Spec (#42) and anyway was never implemented

- there are still 9 issues open in the Spec regarding capture-from-media-element (the Spec also talks about capture-from-canvas, which is shipped in Cr and FF and not the topic of this ITS)

 - in particular, using Promises (#58) in the spec seems to have got a cold reception;

- crrev.com/2727583007 taught the captured stream to follow the source Media Element events, so e.g. if the <video/audio> fires an ended Event, the captured stream is ended as well, etc. which improves the ergonomics of the interactions between source and its captureStream.

The Spec is being reviewed for Candidate Recommendation, so I think it's fair to wait for the TAG comments and if nothing major arises, revive this ITS thread.  We might need some wptests as well.

[Miguel Casas-Sanchez]

I got a few pings asking for shipping this feature and since nothing is blocking it and FF already ships it, it's happening in https://chromium-review.googlesource.com/c/544899  (unless somebody has any objections).

[Chris Harrelson]

Please wait for a third LGTM. We'll respond soon.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUS085Q_CYVGLz8n%3DZWmF14oTQOi%2BdaF8ptLa%3DjYFLM-1jnfA%40mail.gmail.com.

[Miguel Casas-Sanchez]

On 22 June 2017 at 16:46, Chris Harrelson <chri...@chromium.org> wrote:

Please wait for a third LGTM. We'll respond soon.

Oh, I miscounted :P  -- sure, will wait for the third LGTM.

[Rick Byers]

Firefox has only a prefixed implementation and nobody else has an implementation, right?  I.e. we're the first to ship using the actual spec'd API names?

Miguel, I think what Philip was asking was that you go through the open issues (at least those with potential compat implications like this one he filed) and comment saying which behavior Chrome is shipping.  That way at least editors / other implementors can be aware of the decision we're making on these unresolved issues (potentially effectively deciding the issue for the WG, since web compat may prevent us from changing).  Philip, is that right?

It looks like there's been no update on the TAG review since last October when they said there were too many open issues to review the spec.  Once you've done the above, can you comment on the TAG review saying that you believe the spec is in good enough shape to ship this feature in Chrome and they should raise any issues ASAP if they disagree?

What is the status of the web-platform-tests?  I.e. how good is the coverage, and what's the pass rate on your implementation?

Thanks,

   Rick

[Philip Jägenstedt]

On Fri, Jun 23, 2017 at 11:32 PM Rick Byers <rby...@chromium.org> wrote:

Firefox has only a prefixed implementation and nobody else has an implementation, right?  I.e. we're the first to ship using the actual spec'd API names?

Miguel, I think what Philip was asking was that you go through the open issues (at least those with potential compat implications like this one he filed) and comment saying which behavior Chrome is shipping.  That way at least editors / other implementors can be aware of the decision we're making on these unresolved issues (potentially effectively deciding the issue for the WG, since web compat may prevent us from changing).  Philip, is that right?

Yep, that's right.

 

It looks like there's been no update on the TAG review since last October when they said there were too many open issues to review the spec.  Once you've done the above, can you comment on the TAG review saying that you believe the spec is in good enough shape to ship this feature in Chrome and they should raise any issues ASAP if they disagree?

What is the status of the web-platform-tests?  I.e. how good is the coverage, and what's the pass rate on your implementation?

https://github.com/w3c/web-platform-tests/tree/master/mediacapture-fromelement has only a idlharness.js test. Could https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/fast/mediacapturefromelement/ be upstreamed as-is, or do they depend on internals? (I find no internals by grepping.)

 

Thanks,

   Rick

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Miguel Casas]

On 30 June 2017 at 18:21, Philip Jägenstedt <foo...@chromium.org> wrote:

On Fri, Jun 23, 2017 at 11:32 PM Rick Byers <rby...@chromium.org> wrote:

Firefox has only a prefixed implementation and nobody else has an implementation, right?  I.e. we're the first to ship using the actual spec'd API names?

FF is shipping a prefixed implementation indeed.

 

Miguel, I think what Philip was asking was that you go through the open issues (at least those with potential compat implications like this one he filed) and comment saying which behavior Chrome is shipping.  That way at least editors / other implementors can be aware of the decision we're making on these unresolved issues (potentially effectively deciding the issue for the WG, since web compat may prevent us from changing).  Philip, is that right?

Yep, that's right.

 

It looks like there's been no update on the TAG review since last October when they said there were too many open issues to review the spec.  Once you've done the above, can you comment on the TAG review saying that you believe the spec is in good enough shape to ship this feature in Chrome and they should raise any issues ASAP if they disagree?

What is the status of the web-platform-tests?  I.e. how good is the coverage, and what's the pass rate on your implementation?

https://github.com/w3c/web-platform-tests/tree/master/mediacapture-fromelement has only a idlharness.js test. Could https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/fast/mediacapturefromelement/ be upstreamed as-is, or do they depend on internals? (I find no internals by grepping.)

It could be upstreamed, only catch is that we're using MediaRecorder to verify that there's data flowing through the <video/audio>.captureStream(); FF has support for MR, but other browsers pulling the WPTs might be misguided, right?

 

 

Thanks,

   Rick

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUS085Q_CYVGLz8n%3DZWmF14oTQOi%2BdaF8ptLa%3DjYFLM-1jnfA%40mail.gmail.com.

--

Miguel Casas-Sanchez |

Gatopardo del Software |

ydog / mca...@google.com |

+1 (650) 603 1380

[Philip Jägenstedt]

On Tue, Jul 4, 2017 at 8:44 AM Miguel Casas <mca...@google.com> wrote:

On 30 June 2017 at 18:21, Philip Jägenstedt <foo...@chromium.org> wrote:

On Fri, Jun 23, 2017 at 11:32 PM Rick Byers <rby...@chromium.org> wrote:

Firefox has only a prefixed implementation and nobody else has an implementation, right?  I.e. we're the first to ship using the actual spec'd API names?

FF is shipping a prefixed implementation indeed.

 

Miguel, I think what Philip was asking was that you go through the open issues (at least those with potential compat implications like this one he filed) and comment saying which behavior Chrome is shipping.  That way at least editors / other implementors can be aware of the decision we're making on these unresolved issues (potentially effectively deciding the issue for the WG, since web compat may prevent us from changing).  Philip, is that right?

Yep, that's right.

 

It looks like there's been no update on the TAG review since last October when they said there were too many open issues to review the spec.  Once you've done the above, can you comment on the TAG review saying that you believe the spec is in good enough shape to ship this feature in Chrome and they should raise any issues ASAP if they disagree?

What is the status of the web-platform-tests?  I.e. how good is the coverage, and what's the pass rate on your implementation?

https://github.com/w3c/web-platform-tests/tree/master/mediacapture-fromelement has only a idlharness.js test. Could https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/fast/mediacapturefromelement/ be upstreamed as-is, or do they depend on internals? (I find no internals by grepping.)

It could be upstreamed, only catch is that we're using MediaRecorder to verify that there's data flowing through the <video/audio>.captureStream(); FF has support for MR, but other browsers pulling the WPTs might be misguided, right?

If there is no other way to automate tests, then the options are using MediaRecorder or introducing test automation APIs, and in both cases other browsers would need to be some work to get it working. So, in this case I think the dependency on MediaRecorder sounds OK, you could document it in a README perhaps and only change it if you get pushback down the line.

[Rick Byers]

That sounds right to me.  LGTM3 once there's a bug assigned to someone to track getting these tests upstreamed following this plan.

[Miguel Casas-Sanchez]

FTR https://crrev.com/c/566151 upstream the Cr tests.

[Miguel Casas-Sanchez]

I pinged the <video>/<audio> spec issues that are still open.

WPT tests are here, with a minor cleanup PR following and another PR to support moz- prefixed methods in //.

I see several LGTMs, are we good to ship, rbyers@ ?

[Rick Byers]

Thanks for the status update!

Yep, you've got your 3 LGTMs (sorry if that wasn't clear).