"Raw" VP8 Buffer to WebM Container format

[Kyle Cummings]

I currently have developed an Android based application that takes the device's camera frames and converts them to the VP8 format. When I save the VP8 image information, I am saving them in a special format so that the images can be more easily played back later within my program. At the start of the binary file (where the VP8 image is) I store the frame length, the frame type (key / delta), the frame timestamp and then the "raw" VP8 frame buffer. I put "raw" in quotes because I am getting the converted frames from WebRTC, and the library (as far as I can tell) does not put VP8 frames in any sort of file standard format.

So, now that I have the recording and playback working on Android devices, I need to extend that capability to WebRTC capable browsers. However, the VP8 frames will not always be streamed to them over a WebRTC connect. They also need to be downloaded and then played within the browser. From my understanding, there is no interface into any WebRTC enabled browser to have it render specific VP8 frames that are provided outside of a WebRTC connection. However, I was told by the other members of the project that we can achieve this non-p2p playback of VP8 frames if we packed the original VP8 frame to a WebM container before storing them.

I have spent the day looking around the documentation and code, and have had only a little success (the frame files are recognized as media / video files within Ubuntu). I am rather certain that they do not play because I haven't fully / correctly implemented the container wrapping.

So, I was wondering if anyone knew how I can take the "raw" VP8 buffers given to me by WebRTC and then wrap them in a WebM container?

Thanks in advance.

[John Koleszar]

vpxenc from libvpx[1] has a primitive webm writer for writing
video-only webm files. libwebm[2] is a more complete webm writer with
a C++ interface. There are android jni bindings available for it[3].

[1]: http://git.chromium.org/gitweb/?p=webm/libvpx.git
[2]: http://git.chromium.org/gitweb/?p=webm/libwebm.git
[3]: http://git.chromium.org/gitweb/?p=webm/bindings.git

You also may be interested in the Media Source Extensions[4], which
would let you parse your file format and pass it to the video tag
through javascript.
[4]: https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html

> --
> You received this message because you are subscribed to the Google Groups
> "WebM Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to webm-discuss...@webmproject.org.
> To post to this group, send email to webm-d...@webmproject.org.
> Visit this group at
> http://groups.google.com/a/webmproject.org/group/webm-discuss/.
> For more options, visit
> https://groups.google.com/a/webmproject.org/groups/opt_out.
>
>

[Kyle Cummings]

John,

Thanks for the reply. I have been looking at your suggestions and I have one question. I have read through the file vpxenc.c file you mentioned, and I only see code that takes either an I420 or a Y4M frame and encodes that to VP8. Did I miss something?

Thanks again.

[Kyle Cummings]

John, 

Nevermind, I think I might have just found it.

Thanks again for the help.

[John Koleszar]

Hi Kyle,

Take a look at these functions:

write_webm_seek_element(EbmlGlobal *ebml, unsigned long id, off_t pos) {
write_webm_seek_info(EbmlGlobal *ebml) {
write_webm_file_header(EbmlGlobal *glob,
write_webm_block(EbmlGlobal *glob,
write_webm_file_footer(EbmlGlobal *glob, long hash) {

[Kyle Cummings]

John, 

I just wanted to write back and thank you again for your help. I was able to figure things out from the files you suggested to me. So now I have my own WebM files being built from WebRTC VP8 frames, and then being played back in the browser.

Thanks again.

Kyle

[Rūdolfs Bundulis]

Just to let you know - ffmpeg also has vp8/webm support so if you have raw yuv frames, it might be easier to use ffmpeg, at least i've also succeded to stream live video via HTTP with ffmpeg and the code for encoding was very compact.

Best Regards,

Rudolfs Bundulis

2013/7/1 Kyle Cummings <cumming...@gmail.com>

[John Koleszar]

Glad to hear it, Kyle. Happy to help.

John