Intent to Ship: Media Stream Recording API

[Miguel Casas-Sanchez]

Contact emails: nik...@chromium.org, mca...@chromium.org

Spec: http://w3c.github.io/mediacapture-record/MediaRecorder.html
Implement DD:  https://goo.gl/vSjzC5

Summary:

"An API for media stream recording. Allows web applications to record encoded media streams." (from the ITI).

Intent-to-implement link:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/2l_G_apqk30

Supported platforms: ATM all except Android.
Android is not supported due to third_party/libwebm [1] not compiled for it.

Demo link:
https://webrtc.github.io/samples/src/content/getusermedia/record/

Debuggability: no specific comments here.

Interoperability and Compatibility Risk:

The API has been shipped in Firefox for a few years now (2?). Most of the discussion/github bugs relate to the API to indicate the bitrate for the video encoder, which seems low risk, if any.

OWP launch tracking bug: http://crbug.com/261321.

Entry on the feature dashboard: https://www.chromestatus.com/features/5929649028726784



[1] https://code.google.com/p/chromium/codesearch#chromium/src/third_party/libwebm/&q=libwebm&sq=package:chromium

[Niklas Enbom]

A clarification on this, we're not planning to skip Android support all together. The current plan is do add it to desktop first and get some usage before taking a binary size hit on Android (one version later or so).  If API owners thinks differently we can add it immediately instead.

[Elliott Sprehn]

It's nice to present a unified API across our version numbers, it makes it easier for developers to reason about the capabilities of Chrome. If the only thing blocking us is binary size, and we're going to do it soon anyway, I say let's go for it now. If the binary size increase isn't acceptable today we should rethink shipping the API on other platforms until we have a strategy to ship everywhere. :)

What's the increase?

[Miguel Casas]

On Nov 26, 2015 12:39 PM, "Elliott Sprehn" <esp...@chromium.org> wrote:

>

> It's nice to present a unified API across our version numbers, it makes it easier for developers to reason about the capabilities of Chrome. If the only thing blocking us is binary size, and we're going to do it soon anyway, I say let's go for it now. If the binary size increase isn't acceptable today we should rethink shipping the API on other platforms until we have a strategy to ship everywhere. :)

>

> What's the increase?

Last time I checked it was 12KB in Debug, 10KB in Release. (in my Linux box).

[Elliott Sprehn]

That doesn't seem bad for the added functionality and compatability with other browsers. I'm sure we'll get 10k back as we simplify the codebase post merge. :)

Can we get a UseCounter for this feature before it ships though? It's nice to track adoption.

[Rick Byers]

I take it that Travis's editorship of the spec indicates some support from Microsoft for the API in its current form?  I see the Chrome Status entry says "public support" (with just a link to the spec), but status.modern.ie says only "under consideration".

I agree with Elliott's characterization around Android entirely.  If binary size on Android is a major issue, then better to figure that out now (if we'd NEVER be willing to take the hit, then that could impact our decision to ship this API anywhere!).

Rick

[Miguel Casas]

On 26 November 2015 at 14:08, Elliott Sprehn <esp...@chromium.org> wrote:

That doesn't seem bad for the added functionality and compatability with other browsers. I'm sure we'll get 10k back as we simplify the codebase post merge. :)

I'll patch together a CL adding third_party/libwebm and the MediaRecorder classes to Cr Android builds. Is it safe to assume that, aside from this issue, the rest of this ITS looks OK?

Can we get a UseCounter for this feature before it ships though? It's nice to track adoption.

Yeah! There's an UMA timeline in https://goo.gl/fHafeS :)

Any other trackers I {c,sh}ould add?

--

Miguel Casas-Sanchez |

Gatopardo del Software |

ydog / mca...@google.com |

+1 (650) 603 1380

[Rick Byers]

On Mon, Nov 30, 2015 at 1:53 PM, 'Miguel Casas' via blink-dev <blin...@chromium.org> wrote:

On 26 November 2015 at 14:08, Elliott Sprehn <esp...@chromium.org> wrote:

That doesn't seem bad for the added functionality and compatability with other browsers. I'm sure we'll get 10k back as we simplify the codebase post merge. :)

I'll patch together a CL adding third_party/libwebm and the MediaRecorder classes to Cr Android builds. Is it safe to assume that, aside from this issue, the rest of this ITS looks OK?

It looks OK to me (though I'd still like to hear how Travis or other Edge folks feel about this API). 

[Miguel Casas]

On 30 November 2015 at 10:53, Miguel Casas <mca...@google.com> wrote:

On 26 November 2015 at 14:08, Elliott Sprehn <esp...@chromium.org> wrote:

That doesn't seem bad for the added functionality and compatability with other browsers. I'm sure we'll get 10k back as we simplify the codebase post merge. :)

I'll patch together a CL adding third_party/libwebm and the MediaRecorder classes to Cr Android builds. Is it safe to assume that, aside from this issue, the rest of this ITS looks OK?

FTR: https://crrev.com/1486623004 . Increase in size is 21KB (10KB for libwebm and another 11KB for the classes).

[TAMURA, Kent]

LGTM1 if we ship it for all platforms at the same time.

--

TAMURA Kent
Software Engineer, Google

[Philip Jägenstedt]

Which are the supported recording formats? In the design doc and this thread I see WebM and VP9 mentioned, so is a WebM container with VP9+Vorbis the only format, or is VP8 also supported? Any other containers?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Miguel Casas]

The supported video codecs are VP8 and VP9. The only supported audio codec is Opus. The only container format currently is Webm (simplified Matroska) in streaming format.

MediaRecorder.canRecordMimeType("bla/foo") [1] will answer "maybe" if it's supported, and currently it says so for video/vp8, video/vp9, audio/opus, video/webm and audio/webm.

[1] https://w3c.github.io/mediacapture-record/MediaRecorder.html#widl-MediaRecorder-canRecordMimeType-DOMString-DOMString-mimeType

[David Dorwin]

On Wed, Dec 2, 2015 at 9:26 AM, 'Miguel Casas' via blink-dev <blin...@chromium.org> wrote:

The supported video codecs are VP8 and VP9. The only supported audio codec is Opus. The only container format currently is Webm (simplified Matroska) in streaming format.

MediaRecorder.canRecordMimeType("bla/foo") [1] will answer "maybe" if it's supported, and currently it says so for video/vp8, video/vp9, audio/opus, video/webm and audio/webm.

Do you mean 'video/webm; codecs="vp8"' and similar? 'video/vp8' is not a valid MIME type.

The spec says "A MimeType, including parameters..." so using the codecs parameter seems correct. The spec should probably be more explicit and reference other definitions, such as http://www.w3.org/TR/html5/embedded-content-0.html#mime-types and/or RFC(s).

[Philip Jägenstedt]

On Wed, Dec 2, 2015 at 8:14 PM, David Dorwin <ddo...@chromium.org> wrote:

On Wed, Dec 2, 2015 at 9:26 AM, 'Miguel Casas' via blink-dev <blin...@chromium.org> wrote:

The supported video codecs are VP8 and VP9. The only supported audio codec is Opus. The only container format currently is Webm (simplified Matroska) in streaming format.

MediaRecorder.canRecordMimeType("bla/foo") [1] will answer "maybe" if it's supported, and currently it says so for video/vp8, video/vp9, audio/opus, video/webm and audio/webm.

Do you mean 'video/webm; codecs="vp8"' and similar? 'video/vp8' is not a valid MIME type.

The spec says "A MimeType, including parameters..." so using the codecs parameter seems correct. The spec should probably be more explicit and reference other definitions, such as http://www.w3.org/TR/html5/embedded-content-0.html#mime-types and/or RFC(s).

It looks like I filed some spec issues a long time ago, including one about canRecordMimeType. I don't think we should ship an API that's inconsistent with MediaSource.isTypeSupported() without good cause.

There are also a number of issues linked from the source code that should be sorted out before shipping.

Finally, the BlobEvent constructor arguments are missing, and in Blink the init dict was made optional in the meantime. That doesn't really work, because BlobEvent.data is non-nullable, so the dict argument should be required, and the data argument should be required and non-nullable.

Can you poke Travis, or can you promote yourself to spec editor and fix these issues? Unless some of the spec issues are about fundamental design issues, it should not be much work.

Philip

[Philip Jägenstedt]

Oh... this has already shipped in Firefox. That rules out certain bikeshedding, but also means we need to be a lot more careful about interop. A quick glance at their IDL reveals that their constructor takes a dictionary and not a string as the second argument, they have another constructor that takes an AudioNode, and other small differences. No MediaRecorderErrorEvent, so what is their error event? One nice thing is that they haven't shipped canRecordMimeType, but they do have the mimeType attribute :/

Are you in contact with the Gecko developers to get sort out these differences? Git blames mostly Randy Lin <rl...@mozilla.com>.

Philip

[Elliott Sprehn]

Hmm.. seems like we should delay shipping until our implementation is closer to Firefox. :)

[Philip Jägenstedt]

Yeah, getting the boring little details in order first would be good. It shouldn't take much time if all implementors and spec editors are responsive.

[Miguel Casas]

Based on Philip and other reviewers' comments (thanks!), the code works now in Android [1], and has caught up [2,3] to the cleaned up Spec [4] (*).

LGTMs then? :)

[1] http://crrev.com/1486623004

[2] http://crrev.com/1497883002
[3] http://crrev.com/1507183002

[4] http://w3c.github.io/mediacapture-record/MediaRecorder.html

(*) Still opened the minor issue of how to call the MediaRecorder Error Event.

[Philip Jägenstedt]

Thanks for addressing the feedback, Miguel!

What is the plan for https://github.com/w3c/mediacapture-record/issues/10? If we ship with "mimeType" then I don't think it'd make sense to ever change the spec.

LGTM2 in any case, it would not make sense to delay shipping by a milestone over a naming issue, but I hope that an explicit decision is made.

[Rick Byers]

LGTM3

[PhistucK]

One thing to note is that a bug (or a feature request?) prevents Media Stream Recording API and Web Audio integration -

crbug.com/482934

Web Audio does not support Opus, but Media Stream Recording records audio as Opus.

☆PhistucK

[Philip Jägenstedt]

Opus support for decodeAudioData would be great, hope someone can take a look at that :)

[Miguel Casas]

Update: the latest Editor Draft [1] includes two added readonly attributes: audioBitsPerSecond and videoBitsPerSecond. CL [2] proposes landing them in Blink.

Rationale: MediaRecorderOptions has several ways of specifying desired audio/video/overall encoded bit rates. The UA has certain latitude in choosing how to deal with those desires including ignoring, truncating, rounding, and/or dividing the overall budget among video and audio as it sees fit. Hence read-back attributes were proposed and added to the spec. 

[1] http://w3c.github.io/mediacapture-record/MediaRecorder.html

[2] http://crrev.com/1652373004

[Philip Jägenstedt]

I've been looking at the spec and CL, and support shipping this without another intent thread, and it fits under the https://www.chromestatus.com/features/5929649028726784 umbrella. Objections?

[Chris Harrelson]

Fine by me.