Re: Segmenting WEBM Videos

[Tom Finegan]

On Thu, Apr 11, 2013 at 1:50 AM, <roberto...@gmail.com> wrote:

Hi,

I'm playing around with streaming WEBM videos and am trying to understand a problem I have encountered.

I am splitting a test webm video into chunks and feeding them to a HTML5 video tag. This works fine if the client initiates the stream and picks up/plays the first chunk. The video will play and everything is rosy! I am using the following code to play each chunk...

var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');  

sourceBuffer.append(...

However, if the client misses the first chunk or I try to play the video from say 5 chunks in, the video does not play and I get the following error in the console...

Your first "chunk" contains the following WebM (Matroska) container elements:

- EBML Header

- Segment Info

- Tracks

The client player needs that information to demux, decompress, and play back the stream(s) in the container. Whatever acts as your server will need to cache the above information. Each time a client connects, the server will need to send that initial setup chunk, and then it can begin sending the newest available data. 

Basically:

- Your first chunk should contain only the WebM elements noted above. A partial element following those elements could (probably will) break a parser.

- Subsequent chunks should start on WebM cluster boundaries.

 

INVALID_STATE_ERR: DOM Exception 11 

I assume here that my chunks are not valid or that I am missing some data that the video tag needs to play the video. I assume this data is in the first chunk? 

I'm a noob to video encoding etc... so am struggling a bit to understand (never mind fix!) the problem. 

I found this link...

http://www.ioncannon.net/utilities/1515/segmenting-webm-video-and-the-mediasource-api/ 

But I'm not really sure how to use this or if its even relevant. 

Is my issue down to keyframes and where I am segmenting the video?

I suppose what I am asking is how can I get the video tag to play any chunk of a WEBM video it is given?

Ultimately what I want to do is have the server start streaming the video and each client will pick up the video from whatever point it connects at. i.e. 30 minutes in or whatever. I am building this into a Node.js project that uses websockets to stream binary data. I have everything working with node & websockets when I stream the video from the start. But if a client connects to a server that is already broadcasting a stream it fails as above.

Thanks

--
You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webm-discuss...@webmproject.org.
To post to this group, send email to webm-d...@webmproject.org.
Visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
For more options, visit https://groups.google.com/a/webmproject.org/groups/opt_out.
 
 

[roberto...@gmail.com]

That makes sense!

How do I find the above elements? Is there a number of bytes I should look for or segment up to? 

i.e. first segment is up until I encounter a certain sequence of bytes?

Thanks for the feedback by the way

[roberto...@gmail.com]

hhhmmm,

as a quick and dirty test I tried having the client play the first chunk and then skip the next few chunks and resume playing from the 5th chunk on.

it plays the first chunk but then dies as soon as it tries to play the 5th chunk with...

Uncaught Error: SyntaxError: DOM Exception 12

Should this have worked as the video tag received and played the first chunk which would have contained the container elements?

[Tom Finegan]

Unless you're certain that:

- your first chunk contained only the elements I mentioned up thread, and

- there isn't a partial element following those setup elements within your first chunk

You'd have to get very lucky- i.e. your chunk size just happened to be exactly the length of the setup elements for your test file.

Now assuming you were lucky, your luck will run out almost immediately. Cluster length is going to vary from cluster to cluster. If you don't do any parsing, and you skip a chunk, subsequent chunks will start in (semi) random places within each WebM cluster. I suspect that's the source of your SyntaxError.

Getting back to the observed behavior: It sounds like you weren't lucky. The chunk size you used resulted in sending the initial setup elements, and at least a portion of the first cluster in the WebM file. Once that partial cluster was sent down to the media source element, assuming you didn't give the MSE the remaining data belonging to that cluster, I don't think parsing would ever recover. 

To your earlier question: There are many WebM parsing solutions available. Which one you use depends on your needs. Libwebm supports parsing of WebM files, but it is written in C++, which may not be what you want to use in your server environment. I know there's at least one javascript based WebM parser on GitHub; more are probably out there. You could also write your own parser using the information in the Matroska specification, and the WebM container guidelines if that's the way you wanted to go.

[roberto...@gmail.com]

Thanks for all the feedback Tom. Its really helpful.

I'll look into the parsing side of it but I do have a question from your explanation of clusters etc...

I thought I read somewhere that WEBM was designed for streaming. That, theoretically you could take a chunk of the file and it should play?

It sounds to me like I need to be splitting the file on whole clusters otherwise I can't rely on it being properly played back. Is that correct??

On Thursday, April 11, 2013 9:50:36 AM UTC+1, roberto...@gmail.com wrote:

Hi,

I'm playing around with streaming WEBM videos and am trying to understand a problem I have encountered.

I am splitting a test webm video into chunks and feeding them to a HTML5 video tag. This works fine if the client initiates the stream and picks up/plays the first chunk. The video will play and everything is rosy! I am using the following code to play each chunk...

var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');  

sourceBuffer.append(...

However, if the client misses the first chunk or I try to play the video from say 5 chunks in, the video does not play and I get the following error in the console...

[Tom Finegan]

On Sun, Apr 14, 2013 at 2:00 PM, <roberto...@gmail.com> wrote:

Thanks for all the feedback Tom. Its really helpful.

I'll look into the parsing side of it but I do have a question from your explanation of clusters etc...

I thought I read somewhere that WEBM was designed for streaming. That, theoretically you could take a chunk of the file and it should play?

It sounds to me like I need to be splitting the file on whole clusters otherwise I can't rely on it being properly played back. Is that correct??

I'm not familiar with the original design goals of the Matroska container. WebM is certainly intended to allow for streaming, but I wouldn't go as far as to say it was designed for it. 

Grabbing a chunk of a WebM container bitstream at a random location is not something that's expected to work:

- Vorbis audio decode/playback requires setup data contained in the TrackEntry (stored within the CodecPrivate)

- VP8 playback must begin on a key frame (though some players are smart enough to wait for a keyframe to appear before attempting to decode video)

When I was working on webmlive, I chose to send WebM data to the server as chunks of two types:

- Metadata (contained EBML header, segment info, tracks)

- Clusters

The server would cache the metadata chunk, and when new player clients connected it would send the metadata chunk followed by the newest received cluster from the encoder client. The decision to go this route with the chunks was partly rooted in my understanding that the the Cluster element in a WebM container is the largest storage unit that will not depend[1] on another element within the container.

So, getting back to your final question: Yes, but even using whole clusters has the one sticking point noted in [1].

[1] This requires that the first video frame in each cluster is always a keyframe, or that the player receiving chunks is smart enough to wait until a keyframe is available to start video decoding.

On Thursday, April 11, 2013 9:50:36 AM UTC+1, roberto...@gmail.com wrote:

Hi,

I'm playing around with streaming WEBM videos and am trying to understand a problem I have encountered.

I am splitting a test webm video into chunks and feeding them to a HTML5 video tag. This works fine if the client initiates the stream and picks up/plays the first chunk. The video will play and everything is rosy! I am using the following code to play each chunk...

var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');  

sourceBuffer.append(...

However, if the client misses the first chunk or I try to play the video from say 5 chunks in, the video does not play and I get the following error in the console...

INVALID_STATE_ERR: DOM Exception 11 

I assume here that my chunks are not valid or that I am missing some data that the video tag needs to play the video. I assume this data is in the first chunk? 

I'm a noob to video encoding etc... so am struggling a bit to understand (never mind fix!) the problem. 

I found this link...

http://www.ioncannon.net/utilities/1515/segmenting-webm-video-and-the-mediasource-api/ 

But I'm not really sure how to use this or if its even relevant. 

Is my issue down to keyframes and where I am segmenting the video?

I suppose what I am asking is how can I get the video tag to play any chunk of a WEBM video it is given?

Ultimately what I want to do is have the server start streaming the video and each client will pick up the video from whatever point it connects at. i.e. 30 minutes in or whatever. I am building this into a Node.js project that uses websockets to stream binary data. I have everything working with node & websockets when I stream the video from the start. But if a client connects to a server that is already broadcasting a stream it fails as above.

Thanks

--

[roberto...@gmail.com]

this all pretty much ties in with what I have been seeing too.

I suppose what I really wanted was to get a better understanding of WEBM and you've definitely helped with that.

One final question, can you recommend a good tool or application for breaking up webm files and inspecting them? I'd like to take a sample file, extract its metadata and inspect its clusters. 

Thanks

[Stefan Lederer]

Hi,

 

we did it a while ago: http://www-itec.uni-klu.ac.at/dash/?page_id=746#contentgen

 

But I haven’t tried it since a while with webm … and as the Media Source Extensions API changed a lot I assume that you might have to do some modifications on the scripts.

 

Best regards,

Stefan

[roberto...@gmail.com]

Thanks Stefan.

[roberto...@gmail.com]

Hey,

No I didn't find a solution. I didn't really go much further with parsing the WEBM file.

If you get somewhere I'd be very interested to hear.

Robert

On Tuesday, May 21, 2013 10:15:22 PM UTC+1, spiagg...@gmail.com wrote:

Roberto, have you find a solution for the first webm chunk issue? I'm also interested!

Il giorno martedì 16 aprile 2013 12:36:56 UTC+2, roberto...@gmail.com ha scritto:

Thanks Stefan.

[asher...@gmail.com]

I'd be interested too, I need to prepare the first chunk and then use that for new connections to the http server that supplies a live feed... anyhelp on how to assemble that chunk much appreciated... I know what needs to go in it, but not the format, and ultimately the hex...

A

[ademola onabanjo]

@asher did you later find a solution? am also having this exact issue