Media Source Extensions ("AppendBytes API")

[Hadar]

Hello,

There is a new interesting API that already introduced in Google
Chrome:
http://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.html

It gives the application-level javascript an API that can help
implementing ad insertion, adaptive streaming, video editing and more.
The developer can actually take an HTML5 video buffer (segment) and
append it to the playback on Javascript. Do you plan to support this
API? What API websites' client side developers have/will have to
achieve similar functionality?

Thanks,
Hadar Weiss

[Anant Narayanan]

On 4/22/12 5:41 AM, Hadar wrote:
> It gives the application-level javascript an API that can help
> implementing ad insertion, adaptive streaming, video editing and more.
> The developer can actually take an HTML5 video buffer (segment) and
> append it to the playback on Javascript. Do you plan to support this
> API? What API websites' client side developers have/will have to
> achieve similar functionality?

It seems like most of these use cases can be tackled with simple
MediaStreams and ProcessedMediaStreams. For ad insertion, as an example,
one may pause the incoming MediaStream, play the ad, and resume the
original stream.

I'm not aware of any current plans to implement that API.

Speaking for myself, I'm not a fan of appendBytes-like APIs in general,
they are mostly a replication of the Flash equivalent (which many are
familiar with since it was the only way to deliver video for a while). I
think the use-cases can (and if not, should) be fulfilled with APIs that
are more "webby", and that play well with all the existing HTML media
elements.

-Anant

[Timothy B. Terriberry]

Anant Narayanan wrote:
> Speaking for myself, I'm not a fan of appendBytes-like APIs in general,
> they are mostly a replication of the Flash equivalent (which many are

In addition to Anant's objections, the last time I looked at this API,
the actual format of the bytes you could append was not specified in
enough detail to allow interoperable implementations, some of the
details essentially required you to parse Ogg resources in the JS before
you could hand them off properly (if you were using that format), the
seeking model had issues (and I believe could not work at all with Ogg),
and it was very unclear how it was meant to interact with the browser's
cache.

That was over a year ago, though, so some of this may have been
addressed, but if so that'd be news to me.

[Robert O'Callahan]

On Tue, Apr 24, 2012 at 7:49 AM, Timothy B. Terriberry <
tterr...@mozilla.com> wrote:

> In addition to Anant's objections, the last time I looked at this API, the
> actual format of the bytes you could append was not specified in enough
> detail to allow interoperable implementations, some of the details
> essentially required you to parse Ogg resources in the JS before you could
> hand them off properly (if you were using that format), the seeking model
> had issues (and I believe could not work at all with Ogg), and it was very
> unclear how it was meant to interact with the browser's cache.
>

I think those issues have been improved in the latest draft.
http://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.html

I actually think Media Source is a reasonably good spec for situations
where the app has to manipulate compressed data buffers --- e.g. for
script-controlled adaptive streaming. Yes, the script has to know about the
container format, so that adds a lot of complexity that we should avoid
with higher-level APIs when possible (e.g. if we just want to play two
resources in series). But sometimes it's not possible.

Rob
--
“You have heard that it was said, ‘Love your neighbor and hate your enemy.’
But I tell you, love your enemies and pray for those who persecute you,
that you may be children of your Father in heaven. ... If you love those
who love you, what reward will you get? Are not even the tax collectors
doing that? And if you greet only your own people, what are you doing more
than others?" [Matthew 5:43-47]

[wha...@gmail.com]

On Tuesday, April 24, 2012 6:18:07 PM UTC+3, Robert O&#39;Callahan wrote:
> On Tue, Apr 24, 2012 at 7:49 AM, Timothy B. Terriberry &lt;
> tterr...@mozilla.com&gt; wrote:
>
> &gt; In addition to Anant&#39;s objections, the last time I looked at this API, the
> &gt; actual format of the bytes you could append was not specified in enough
> &gt; detail to allow interoperable implementations, some of the details
> &gt; essentially required you to parse Ogg resources in the JS before you could
> &gt; hand them off properly (if you were using that format), the seeking model
> &gt; had issues (and I believe could not work at all with Ogg), and it was very
> &gt; unclear how it was meant to interact with the browser&#39;s cache.
> &gt;

>
> I think those issues have been improved in the latest draft.
> http://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.html
>
> I actually think Media Source is a reasonably good spec for situations
> where the app has to manipulate compressed data buffers --- e.g. for
> script-controlled adaptive streaming. Yes, the script has to know about the
> container format, so that adds a lot of complexity that we should avoid
> with higher-level APIs when possible (e.g. if we just want to play two

> resources in series). But sometimes it&#39;s not possible.

>
> Rob
> --
> “You have heard that it was said, ‘Love your neighbor and hate your enemy.’
> But I tell you, love your enemies and pray for those who persecute you,
> that you may be children of your Father in heaven. ... If you love those
> who love you, what reward will you get? Are not even the tax collectors
> doing that? And if you greet only your own people, what are you doing more

> than others?&quot; [Matthew 5:43-47]

Hi,

Any news considering this?
There are other scenarios that require this js-level API.
In our case, we don't get the video from a single server but fetch it dynamically using javascript. We need an API that will allow us to push this data to the video tag...

Thanks,
Hadar

[ezz...@gmail.com]

Just to say meh, "me too".

Also, no it's enough for javascript to have only a vaguely superficial idea about the format. Webserver should decide on that anyway. Going without appendBytes is about losing freedom - webmasters are now forced to wait for that each particular feature to ramp up enough traction to have it pushed upstream.

FYI, Pandora uses appendBytes to play mp3s. Best thing about it is when evilcorp implements DRM that way, it's a trivial place to place the tap to get
clean stream. The positive face of this is of course RTMFP-style controlled P2P object replication.

For example adaptive streaming. Adobe gives me rather embarassing protocol to talk to (RTMP), or just lets me write the stream. Hurray! Forcibly switching to different transcode can be matter of single H264 NAL. All done in HTML5, except the bloody Flash display window itself. But only javascript's idea about this is to strip first 13 bytes from every request except the first.

[Chris Double]

On Sat, Oct 27, 2012 at 7:51 AM, <ezz...@gmail.com> wrote:
> Just to say meh, "me too".

The MediaSource API is being worked on. Bug 778617 is the bug for it:

<https://bugzilla.mozilla.org/show_bug.cgi?id=778617>

--
http://www.bluishcoder.co.nz