Status of HLS and MPEG-2 Transport Streams

[Henri Sivonen]

I noticed some past discussion about HLS and MPEG-2 Transport Streams.
Is https://wiki.mozilla.org/Platform/MediaSourceExtensions the current
source of truth and we aren't supporting either HLS or MPEG-2 TS--not
even on Firefox OS?

Does an implementation of HLS in JS using MSE exist?

--
Henri Sivonen
hsiv...@hsivonen.fi
https://hsivonen.fi/

[Gary Chen]

Hi Henri,
I did some HLS research and experiment before.
As I know we can parse m3u8 files in JS and then pass streaming segment to MSE.
According to the spec [1], mp2t format is listed on MSE specification, but we still under legal discussion on bug 990880[2].

So if platform(gecko) can not support mp2t,
we might need to implement conversion of mp2t video chunks to MPEG-4 stream format and then use MSE.

[1] https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/byte-stream-format-registry.html
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=990880

Best regards
-------------------------------------
Gary Chen 陳柏宇
Mozilla Taiwan

T: +886-2-87861100 # 206
M: +886-975151712
gc...@mozilla.com

> _______________________________________________
> dev-media mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-media
>

[Henri Sivonen]

On Wed, Jun 4, 2014 at 12:22 PM, Gary Chen <gc...@mozilla.com> wrote:
> As I know we can parse m3u8 files in JS and then pass streaming segment to MSE.
> According to the spec [1], mp2t format is listed on MSE specification, but we still under legal discussion on bug 990880[2].

Thanks. What other browsers, if any, support MPEG-2 TS with MSE?

[Ralph Giles]

On 2014-06-04 1:25 AM, Henri Sivonen wrote:
> I noticed some past discussion about HLS and MPEG-2 Transport Streams.
> Is https://wiki.mozilla.org/Platform/MediaSourceExtensions the current
> source of truth and we aren't supporting either HLS or MPEG-2 TS--not
> even on Firefox OS?

AFAIK the plan is to support HLS by parsing the mpegts into fragmented
mp4 in js, assuming the mpeg-2 video will be accepted by the h.264
decoder, as Gary described. In particular we're going to write support
for this into the Gaia media player.

If that doesn't work, or is too slow, the fallback plan is to use a
native implementation, but limit it to privileged apps.

In both cases we really want to avoid having to support mpeg-2 ts and
HLS directly in the web platform.

> Does an implementation of HLS in JS using MSE exist?

There are bits and pieces (e.g. video.js) I've not seen a complete
library like there is for DASH.

-r

[Gary Chen]

Comment inline.

Thanks~

----- Original Message -----
> From: "Ralph Giles" <gi...@mozilla.com>
> To: dev-...@lists.mozilla.org
> Sent: Thursday, June 5, 2014 1:08:53 AM
> Subject: Re: Status of HLS and MPEG-2 Transport Streams
>
> On 2014-06-04 1:25 AM, Henri Sivonen wrote:
> > I noticed some past discussion about HLS and MPEG-2 Transport Streams.
> > Is https://wiki.mozilla.org/Platform/MediaSourceExtensions the current
> > source of truth and we aren't supporting either HLS or MPEG-2 TS--not
> > even on Firefox OS?
>
> AFAIK the plan is to support HLS by parsing the mpegts into fragmented
> mp4 in js, assuming the mpeg-2 video will be accepted by the h.264
> decoder, as Gary described. In particular we're going to write support
> for this into the Gaia media player.

Wow~~ sounds pretty cool~
Is there any bug or wiki page discuss about this feature?

>
> If that doesn't work, or is too slow, the fallback plan is to use a
> native implementation, but limit it to privileged apps.

What is native implementation? Can I ask for more detail?
Thanks~

> In both cases we really want to avoid having to support mpeg-2 ts and
> HLS directly in the web platform.
>
> > Does an implementation of HLS in JS using MSE exist?
>
> There are bits and pieces (e.g. video.js) I've not seen a complete
> library like there is for DASH.
>
> -r
>

[Ralph Giles]

On 2014-06-04 10:36 AM, Gary Chen wrote:

> Wow~~ sounds pretty cool~

It does, doesn't it?

To be clear, when I say "we" I mean "no one one the gecko team". I don't
know who, if anyone, is actually working on this. Sounds like there are
partner requirements for it, so hopefully it's on someone's roadmap.

I don't see a gaia bug. There's some discussion in
https://bugzilla.mozilla.org/show_bug.cgi?id=941351 which is about
supporting HLS m3u8 directly as the source of an HTML video element,
like Apple Safari does.

> What is native implementation? Can I ask for more detail?

Native implementation would be adding an mpeg-ts parser in C++ to gecko
(perhaps the one from android) so video muxed this way can be accepted
directly by the MediaSource Extensions API. We don't want to do this if
we can avoid it. mpeg-ts is a legacy format, and C++ parsers add to our
security and maintenance surface.

The m3u8 parsing should still be done in js: there's no performance
advantage to C++ for that. If we decided to support HLS directly we
would add the same code to gecko as a jsm.

-r

[Ralph Giles]

On 2014-06-04 3:53 AM, Henri Sivonen wrote:
> Thanks. What other browsers, if any, support MPEG-2 TS with MSE?

According to http://www.jwplayer.com/html5/mediasource/ only Safari 8 on
OSX 10.10.

-r

[Sotaro Ikeda]

>AFAIK the plan is to support HLS by parsing the mpegts into fragmented
>mp4 in js, assuming the mpeg-2 video will be accepted by the h.264
>decoder, as Gary described. In particular we're going to write support
>for this into the Gaia media player.

IIRC, we can not support youtube HLS stream by gaia media player by some business reasons, not by a technical reason. On b2g v1.01, youtube video playback was implemented by gaia video app. But it is dropped before production. b2g need to playback youtube video on browser app.

sotaro

----- Original Message -----
From: "Ralph Giles" <gi...@mozilla.com>
To: dev-...@lists.mozilla.org
Sent: Wednesday, June 4, 2014 1:08:53 PM
Subject: Re: Status of HLS and MPEG-2 Transport Streams

On 2014-06-04 1:25 AM, Henri Sivonen wrote:
> I noticed some past discussion about HLS and MPEG-2 Transport Streams.
> Is https://wiki.mozilla.org/Platform/MediaSourceExtensions the current
> source of truth and we aren't supporting either HLS or MPEG-2 TS--not
> even on Firefox OS?

AFAIK the plan is to support HLS by parsing the mpegts into fragmented
mp4 in js, assuming the mpeg-2 video will be accepted by the h.264
decoder, as Gary described. In particular we're going to write support
for this into the Gaia media player.

If that doesn't work, or is too slow, the fallback plan is to use a
native implementation, but limit it to privileged apps.

[Ralph Giles]

On 2014-06-04 12:39 PM, Sotaro Ikeda wrote:

> b2g need to playback youtube video on browser app.

Our MediaSource Extensions implementation should work for the youtube
player on b2g the same as it does desktop.

-r