Intent to Implement & Ship: Support Opus in mp4 (ISO-BMFF) with Media Source Extensions (MSE)

330 megtekintés
Ugrás az első olvasatlan üzenetre

Dale Curtis

olvasatlan,
2018. aug. 16. 17:30:462018. 08. 16.
– blink-dev, Matt Wolenetz
dalec...@chromium.org http://opus-codec.org/docs/opus_in_isobmff.html
Opus is an audio codec, already supported in normal src=<url> Chrome HTML5 playback within the mp4, ogg, and webm containers as well as within the webm container using MSE. This feature adds support for the Opus codec in the mp4 container to MSE. Since the AV1 video codec has adopted the MP4 container, it's important to have a modern, royalty-free audio codec available alongside it. Firefox: Shipped Edge: No public signals (but supports opus in webm) Safari: No public signals (no opus support) Web developers: Positive The spec is listed as incomplete, despite Chrome already having support in src= and Firefox having support in MSE and src=. As such there's always the risk that the spec changes in some future way. None. Yes, with a caveat for encrypted opus content. Android versions prior to Lollipop do not support Opus and we must use the platform decoder for encrypted content. For clear content all versions of Android will have support via Chrome's built-in decoder. https://bugs.chromium.org/p/chromium/issues/detail?id=649438 https://www.chromestatus.com/features/5100845653819392 Yes. https://chromium-review.googlesource.com/c/chromium/src/+/1178929

Yoav Weiss

olvasatlan,
2018. aug. 17. 4:25:452018. 08. 17.
– Dale Curtis, blink-dev, Matt Wolenetz
Can such support (once added) be feature detected?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwf-A2cH-x6YSFqN8c_xdAivEakJoSXcrRFiWwD6G37UQw%40mail.gmail.com.

Dale Curtis

olvasatlan,
2018. aug. 17. 11:45:232018. 08. 17.
– Yoav Weiss, blink-dev, Matt Wolenetz
Yes, through the standard media canPlayType/isTypeSupported mechanisms. E.g., for MSE pages can check MediaSource.isTypeSupported('audio/mp4; codecs="opus"');

- dale
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Yoav Weiss

olvasatlan,
2018. aug. 17. 12:02:392018. 08. 17.
– Dale Curtis, blink-dev, Matt Wolenetz
On Fri, Aug 17, 2018 at 5:45 PM Dale Curtis <dalec...@chromium.org> wrote:
Yes, through the standard media canPlayType/isTypeSupported mechanisms. E.g., for MSE pages can check MediaSource.isTypeSupported('audio/mp4; codecs="opus"');

Sounds good.
 

- dale


On Friday, August 17, 2018, Yoav Weiss <yo...@yoav.ws> wrote:
Can such support (once added) be feature detected?

On Thu, Aug 16, 2018 at 11:30 PM Dale Curtis <dalec...@chromium.org> wrote:
dalec...@chromium.org http://opus-codec.org/docs/opus_in_isobmff.html
Opus is an audio codec, already supported in normal src=<url> Chrome HTML5 playback within the mp4, ogg, and webm containers as well as within the webm container using MSE. This feature adds support for the Opus codec in the mp4 container to MSE. Since the AV1 video codec has adopted the MP4 container, it's important to have a modern, royalty-free audio codec available alongside it. Firefox: Shipped Edge: No public signals (but supports opus in webm) Safari: No public signals (no opus support) Web developers: Positive The spec is listed as incomplete, despite Chrome already having support in src= and Firefox having support in MSE and src=. As such there's always the risk that the spec changes in some future way.

That's unfortunate. Are you active in the standardization process? (Is there a process? Not familiar with opus.org)
Can we make sure that future changes will be backwards compatible? If they won't be, what would be our options?

Not sure if this is a blocker, but I would like to better understand the situation. 
 
None. Yes, with a caveat for encrypted opus content. Android versions prior to Lollipop do not support Opus and we must use the platform decoder for encrypted content. For clear content all versions of Android will have support via Chrome's built-in decoder.

I'm guessing that wherever it's not supported by the OS, isTypeSupported will return false, right?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Dale Curtis

olvasatlan,
2018. aug. 17. 12:19:172018. 08. 17.
– Yoav Weiss, blink-dev, Matt Wolenetz

On Friday, August 17, 2018, Yoav Weiss <yo...@yoav.ws> wrote:

On Fri, Aug 17, 2018 at 5:45 PM Dale Curtis <dalec...@chromium.org> wrote:
Yes, through the standard media canPlayType/isTypeSupported mechanisms. E.g., for MSE pages can check MediaSource.isTypeSupported('audio/mp4; codecs="opus"');

Sounds good.
 

- dale


On Friday, August 17, 2018, Yoav Weiss <yo...@yoav.ws> wrote:
Can such support (once added) be feature detected?

On Thu, Aug 16, 2018 at 11:30 PM Dale Curtis <dalec...@chromium.org> wrote:
dalec...@chromium.org http://opus-codec.org/docs/opus_in_isobmff.html
Opus is an audio codec, already supported in normal src=<url> Chrome HTML5 playback within the mp4, ogg, and webm containers as well as within the webm container using MSE. This feature adds support for the Opus codec in the mp4 container to MSE. Since the AV1 video codec has adopted the MP4 container, it's important to have a modern, royalty-free audio codec available alongside it. Firefox: Shipped Edge: No public signals (but supports opus in webm) Safari: No public signals (no opus support) Web developers: Positive The spec is listed as incomplete, despite Chrome already having support in src= and Firefox having support in MSE and src=. As such there's always the risk that the spec changes in some future way.

That's unfortunate. Are you active in the standardization process? (Is there a process? Not familiar with opus.org)
Can we make sure that future changes will be backwards compatible? If they won't be, what would be our options?

We've reached out to the spec author to see about providing assistance or at least understanding what's incomplete.

There's really not much that's opus specific, so I don't expect complications. I.e., the implementation CL below doesn't even implement the Opus box reading yet (it will before submission), but still play all Opus content (just without preroll or delay working).
 

Not sure if this is a blocker, but I would like to better understand the situation. 
 
None. Yes, with a caveat for encrypted opus content. Android versions prior to Lollipop do not support Opus and we must use the platform decoder for encrypted content. For clear content all versions of Android will have support via Chrome's built-in decoder.

I'm guessing that wherever it's not supported by the OS, isTypeSupported will return false, right?


Yes, but that signaling will be through the EME API, MediaKeysSystemAccess.getConfiguration().
 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Alex Russell

olvasatlan,
2018. aug. 21. 12:31:322018. 08. 21.
– blink-dev, yo...@yoav.ws, wole...@google.com
LGTM1
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Chris Harrelson

olvasatlan,
2018. aug. 21. 14:04:222018. 08. 21.
– Alex Russell, blink-dev, Yoav Weiss, Matt Wolenetz
LGTM2, but please do try to find out more about the status of this spec and whether it can be advanced.

Philip Jägenstedt

olvasatlan,
2018. aug. 21. 14:57:182018. 08. 21.
– Chris Harrelson, Alex Russell, blink-dev, Yoav Weiss, Matt Wolenetz, gi...@mozilla.com, gi...@thaumas.net
LGTM3

The spec is maintained at https://github.com/xiph/opus/blob/master/doc/opus_in_isobmff.html and is mostly recently updated by Xiph veteran Ralph Giles, on CC. Ralph, this is a thread about shipping support for Opus in MP4 in one additional context in Chrome. The spec at http://opus-codec.org/docs/opus_in_isobmff.html claims to be "incomplete", do you have a sense for what it'd take to make it complete?

Philip Jägenstedt

olvasatlan,
2018. aug. 22. 3:56:002018. 08. 22.
– gi...@thaumas.net, Chris Harrelson, Alex Russell, blink-dev, Yoav Weiss, Matt Wolenetz, kin...@flim.org
Dale, do we count as an implementer of that spec, or are we using a third party MP4 demuxer? If we have implemented ourselves, here's a chance to provide spec feedback :)

On Wed, Aug 22, 2018 at 5:17 AM Ralph Giles <gi...@thaumas.net> wrote:
Great news!

The spec is incomplete in that there's been little feedback from
implementors, and afaik no one's gone to the trouble to officially
register the new boxes.

I don't anticipate any changes, but if you find issues, please let me know!

  -r

+ Matthew Gregan who as been overseeing the Firefox implementation.


On 2018-08-21 11:56 AM, Philip Jägenstedt wrote:
> LGTM3
>
> The spec is maintained at
> https://github.com/xiph/opus/blob/master/doc/opus_in_isobmff.html and is
> mostly recently updated by Xiph veteran Ralph Giles, on CC. Ralph, this
> is a thread about shipping support for Opus in MP4 in one additional
> context in Chrome. The spec at
> http://opus-codec.org/docs/opus_in_isobmff.html claims to be
> "incomplete", do you have a sense for what it'd take to make it complete?
>
> On Tue, Aug 21, 2018 at 8:04 PM Chris Harrelson <chri...@chromium.org
> <mailto:chri...@chromium.org>> wrote:
>
>     LGTM2, but please do try to find out more about the status of this
>     spec and whether it can be advanced.
>
>     On Tue, Aug 21, 2018 at 9:31 AM 'Alex Russell' via blink-dev
>     <blin...@chromium.org <mailto:blin...@chromium.org>> wrote:
>
>         LGTM1
>
>         On Friday, August 17, 2018 at 9:19:17 AM UTC-7, Dale Curtis wrote:
>
>
>             On Friday, August 17, 2018, Yoav Weiss <yo...@yoav.ws> wrote:
>
>
>                 On Fri, Aug 17, 2018 at 5:45 PM Dale Curtis
>                 <dalec...@chromium.org> wrote:
>
>                     Yes, through the standard media
>                     canPlayType/isTypeSupported mechanisms. E.g., for
>                     MSE pages can check
>                     MediaSource.isTypeSupported('audio/mp4; codecs="opus"');
>
>
>                 Sounds good.
>
>
>                     - dale
>
>
>                     On Friday, August 17, 2018, Yoav Weiss
>                     <yo...@yoav.ws> wrote:
>
>                         Can such support (once added) be feature detected?
>
>                         On Thu, Aug 16, 2018 at 11:30 PM Dale Curtis
>                         <dalec...@chromium.org> wrote:
>
>                             Contact
>                             emailsdalec...@chromium.orgSpechttp://opus-codec.org/docs/opus_in_isobmff.html
>                             https://www.w3.org/TR/mse-byte-stream-format-isobmff/
>                             SummaryOpus is an audio codec, already

>                             supported in normal src=<url> Chrome HTML5
>                             playback within the mp4, ogg, and webm
>                             containers as well as within the webm
>                             container using MSE. This feature adds
>                             support for the Opus codec in the mp4
>                             container to MSE.MotivationSince the AV1

>                             video codec has adopted the MP4 container,
>                             it's important to have a modern,
>                             royalty-free audio codec available alongside
>                             it. Interoperability riskFirefox: Shipped

>                             Edge: No public signals (but supports opus
>                             in webm) Safari: No public signals (no opus
>                             support) Web developers:
>                             PositiveCompatibility riskThe spec is listed

>                             as incomplete, despite Chrome already having
>                             support in src= and Firefox having support
>                             in MSE and src=. As such there's always the
>                             risk that the spec changes in some future way.
>
>
>                 That's unfortunate. Are you active in the
>                 standardization process? (Is there a process? Not
>                 familiar with opus.org <http://opus.org>)

>                 Can we make sure that future changes will be backwards
>                 compatible? If they won't be, what would be our options?
>
>
>             We've reached out to the spec author to see about providing
>             assistance or at least understanding what's incomplete.
>
>             There's really not much that's opus specific, so I don't
>             expect complications. I.e., the implementation CL below
>             doesn't even implement the Opus box reading yet (it will
>             before submission), but still play all Opus content (just
>             without preroll or delay working).
>
>
>                 Not sure if this is a blocker, but I would like to
>                 better understand the situation.
>
>                             Ongoing technical constraintsNone.Will this

>                             feature be supported on all six Blink
>                             platforms (Windows, Mac, Linux, Chrome OS,
>                             Android, and Android WebView)? Yes or
>                             no.Yes, with a caveat for encrypted opus

>                             content. Android versions prior to Lollipop
>                             do not support Opus and we must use the
>                             platform decoder for encrypted content. For
>                             clear content all versions of Android will
>                             have support via Chrome's built-in decoder.
>
>
>                 I'm guessing that wherever it's not supported by the OS,
>                 isTypeSupported will return false, right?
>
>
>             Yes, but that signaling will be through the EME
>             API, MediaKeysSystemAccess.getConfiguration().
>
>                             OWP launch tracking
>                             bughttps://bugs.chromium.org/p/chromium/issues/detail?id=649438Link
>                             to entry on the Chrome Platform
>                             Statushttps://www.chromestatus.com/features/5100845653819392Requesting
>                             approval to ship?Yes.
>                             https://chromium-review.googlesource.com/c/chromium/src/+/1178929
>
>                             --
>                             You received this message because you are
>                             subscribed to the Google Groups "blink-dev"
>                             group.
>                             To unsubscribe from this group and stop
>                             receiving emails from it, send an email to
>                             blink-dev+...@chromium.org.
>                             To view this discussion on the web visit
>                             https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwf-A2cH-x6YSFqN8c_xdAivEakJoSXcrRFiWwD6G37UQw%40mail.gmail.com

>
>         --
>         You received this message because you are subscribed to the
>         Google Groups "blink-dev" group.
>         To unsubscribe from this group and stop receiving emails from
>         it, send an email to blink-dev+...@chromium.org
>         <mailto:blink-dev+...@chromium.org>.

>         To view this discussion on the web visit

>
>     --
>     You received this message because you are subscribed to the Google
>     Groups "blink-dev" group.
>     To view this discussion on the web visit
>     https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-PfUt%2BYa%3DyQbDRKpeZ8NvpnyozLadpLRD_9qK7cL8o8A%40mail.gmail.com

Dale Curtis

olvasatlan,
2018. aug. 22. 13:58:042018. 08. 22.
– foo...@chromium.org, gi...@thaumas.net, chri...@chromium.org, Alex Russell, blink-dev, Yoav Weiss, Matt Wolenetz, kin...@flim.org
We've implemented our own box reader, https://chromium-review.googlesource.com/c/chromium/src/+/1178929/6/media/formats/mp4/box_definitions.cc -- from the spec side I think it's mostly fine. The biggest issue is that no tooling can generate the mp4 style edit lists necessary for end trimming to be per-spec. I've reached out to Xiph/Firefox folks on another thread about this. I.e., there are no mp4 versions of these files https://people.xiph.org/~greg/opus_testvectors/ -- correctness_trimming_nobeeps.opus being the main issue.

Whether this is a tooling concern or something the spec should address (i.e., by making the end trimming value part of the dOps box), I don't know. 

- dale 

Ralph Giles

olvasatlan,
2018. aug. 22. 20:49:332018. 08. 22.
– Philip Jägenstedt, Chris Harrelson, Alex Russell, blink-dev, Yoav Weiss, Matt Wolenetz, Matthew Gregan
Great news!

The spec is incomplete in that there's been little feedback from
implementors, and afaik no one's gone to the trouble to officially
register the new boxes.

I don't anticipate any changes, but if you find issues, please let me know!

-r

+ Matthew Gregan who as been overseeing the Firefox implementation.

On 2018-08-21 11:56 AM, Philip Jägenstedt wrote:
> emailsdalec...@chromium.orgSpechttp://opus-codec.org/docs/opus_in_isobmff.html
> https://www.w3.org/TR/mse-byte-stream-format-isobmff/
> SummaryOpus is an audio codec, already
> supported in normal src=<url> Chrome HTML5
> playback within the mp4, ogg, and webm
> containers as well as within the webm
> container using MSE. This feature adds
> support for the Opus codec in the mp4
> container to MSE.MotivationSince the AV1
> video codec has adopted the MP4 container,
> it's important to have a modern,
> royalty-free audio codec available alongside
> it. Interoperability riskFirefox: Shipped
> Edge: No public signals (but supports opus
> in webm) Safari: No public signals (no opus
> support) Web developers:
> PositiveCompatibility riskThe spec is listed
> as incomplete, despite Chrome already having
> support in src= and Firefox having support
> in MSE and src=. As such there's always the
> risk that the spec changes in some future way.
>
>
> That's unfortunate. Are you active in the
> standardization process? (Is there a process? Not
> familiar with opus.org <http://opus.org>)
> Can we make sure that future changes will be backwards
> compatible? If they won't be, what would be our options?
>
>
> We've reached out to the spec author to see about providing
> assistance or at least understanding what's incomplete.
>
> There's really not much that's opus specific, so I don't
> expect complications. I.e., the implementation CL below
> doesn't even implement the Opus box reading yet (it will
> before submission), but still play all Opus content (just
> without preroll or delay working).
>
>
> Not sure if this is a blocker, but I would like to
> better understand the situation.
>
> Ongoing technical constraintsNone.Will this
> feature be supported on all six Blink
> platforms (Windows, Mac, Linux, Chrome OS,
> Android, and Android WebView)? Yes or
> no.Yes, with a caveat for encrypted opus
> content. Android versions prior to Lollipop
> do not support Opus and we must use the
> platform decoder for encrypted content. For
> clear content all versions of Android will
> have support via Chrome's built-in decoder.
>
>
> I'm guessing that wherever it's not supported by the OS,
> isTypeSupported will return false, right?
>
>
> Yes, but that signaling will be through the EME
> API, MediaKeysSystemAccess.getConfiguration().
>
> OWP launch tracking
> bughttps://bugs.chromium.org/p/chromium/issues/detail?id=649438Link
> to entry on the Chrome Platform
> Statushttps://www.chromestatus.com/features/5100845653819392Requesting
> approval to ship?Yes.
> https://chromium-review.googlesource.com/c/chromium/src/+/1178929
>
> --
> You received this message because you are
> subscribed to the Google Groups "blink-dev"
> group.
> To unsubscribe from this group and stop
> receiving emails from it, send an email to
> blink-dev+...@chromium.org.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwf-A2cH-x6YSFqN8c_xdAivEakJoSXcrRFiWwD6G37UQw%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwf-A2cH-x6YSFqN8c_xdAivEakJoSXcrRFiWwD6G37UQw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the
> Google Groups "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to blink-dev+...@chromium.org
> <mailto:blink-dev+...@chromium.org>.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/fe029f97-cd0b-4271-86ec-1b93b9504ab7%40chromium.org
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/fe029f97-cd0b-4271-86ec-1b93b9504ab7%40chromium.org?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "blink-dev" group.
> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-PfUt%2BYa%3DyQbDRKpeZ8NvpnyozLadpLRD_9qK7cL8o8A%40mail.gmail.com
> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-PfUt%2BYa%3DyQbDRKpeZ8NvpnyozLadpLRD_9qK7cL8o8A%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>

Dale Curtis

olvasatlan,
2018. aug. 24. 13:22:002018. 08. 24.
– foo...@chromium.org, gi...@thaumas.net, chri...@chromium.org, Alex Russell, blink-dev, Yoav Weiss, Matt Wolenetz, Matthew Gregan
Thanks for review. Given approval and non-blocking concerns with end trimming, we landed support for this in M70:
Válasz mindenkinek
Válasz a szerzőnek
Továbbítás
0 új üzenet