How to decode Alpha channel from VP8 Webm

[abhishek pandey]

Hi, 

Is there any specific API or steps to decode the ALPHA channel from VP8 encoded Webm file. I've done the alpha encoding using webm_tools/alpha_encoder.

Thanks,

Abhishek

[James Zern]

Hi,

On Thu, Oct 1, 2020 at 10:19 AM abhishek pandey <sup....@gmail.com> wrote:

Hi, 

Is there any specific API or steps to decode the ALPHA channel from VP8 encoded Webm file. I've done the alpha encoding using webm_tools/alpha_encoder.

From the decoder's perspective these are independent bitstreams [1]. In the container these are stored as BlockAdditional elements and decoded separately. You can see an example in ffmpeg [2], search for decoder_alpha or has_alpha_channel.

[1] http://wiki.webmproject.org/alpha-channel

[2] https://git.ffmpeg.org/gitweb/ffmpeg.git/blob_plain/HEAD:/libavcodec/libvpxdec.c

 

Thanks,

Abhishek

--
You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webm-discuss...@webmproject.org.
To view this discussion on the web visit https://groups.google.com/a/webmproject.org/d/msgid/webm-discuss/d7cc0fe8-87d0-4e95-a2ae-715cf38423c3n%40webmproject.org.

[abhishek pandey]

After decoding the YUV data, I'm trying to decode the Alpha channel data using this technique(Try and decode buffer->side_data() minus the first 8 bytes as a full frame.)

vpx_codec_decode(&mDecoder, buf, (unsigned int) bytes_in_buffer, NULL, 0)

mVPXImage = vpx_codec_get_frame(&mDecoder, &iter)

Upto here I'm getting the correct YUV image buffer data inside mVPXImage, but alpha is absent.

To deode alpha, I tried below strategy :

vpx_codec_ctx_t mDecoderAlpha;

vpx_codec_err_t error = vpx_codec_dec_init(&mDecoderAlpha, &vpx_codec_vp8_dx_algo, NULL, 0);

vpx_codec_err_t status = vpx_codec_decode(&mDecoderAlpha, buffer + 8, buf_size - 8, nullptr /* user_priv */, 0 /* deadline */);

But i'm getting status as VPX_CODEC_UNSUP_BITSTREAM.

Same thing I tried for vpx_codec_vp9_dx_algo, but same error code is generated.

Any help would be appreciated.

Thanks,

Abhishek

[abhishek pandey]

Thanks James.

I tried the approach mentioned by you(ffmpeg reference implementation). The same approach is used in Chromium project. https://source.chromium.org/chromium/chromium/src/+/master:media/filters/vpx_video_decoder.cc;drc=b9bd197093c14555ab45181bcbae472497b2d324;l=403

But whenever I try to decode the (buffer - 8) bytes as alpha frame data, it throws error VPX_CODEC_UNSUP_BITSTREAM. 

I verified my webm input file in chrome browser, where it is played with transparent BG. 

Also, I tried with the alpha webm input provided in chromium resources : 

https://developers.google.com/web/updates/2013/07/Alpha-transparency-in-Chrome-video#helpful_resources

https://cs.chromium.org/chromium/src/media/test/data/bear-vp8a.webm.

What could be the possible error here, if you have any idea about this?

[James Zern]

On Tue, Oct 6, 2020 at 1:59 PM abhishek pandey <sup....@gmail.com> wrote:

After decoding the YUV data, I'm trying to decode the Alpha channel data using this technique(Try and decode buffer->side_data() minus the first 8 bytes as a full frame.)

vpx_codec_decode(&mDecoder, buf, (unsigned int) bytes_in_buffer, NULL, 0)

mVPXImage = vpx_codec_get_frame(&mDecoder, &iter)

Upto here I'm getting the correct YUV image buffer data inside mVPXImage, but alpha is absent.

To deode alpha, I tried below strategy :

vpx_codec_ctx_t mDecoderAlpha;

vpx_codec_err_t error = vpx_codec_dec_init(&mDecoderAlpha, &vpx_codec_vp8_dx_algo, NULL, 0);

vpx_codec_err_t status = vpx_codec_decode(&mDecoderAlpha, buffer + 8, buf_size - 8, nullptr /* user_priv */, 0 /* deadline */);

What's producing the side data in this case? In the ffmpeg case there's a header with the side data id, but if you're getting the block additional by other means you may not have the same format. VP8 has a frame signature for key frames, which the stream should start on. If you're able to debug this and look at the bytes passed to the alpha decoder you should see '9d' '01' '2a' after the first 3 bytes of the frame.

 

But i'm getting status as VPX_CODEC_UNSUP_BITSTREAM.

Same thing I tried for vpx_codec_vp9_dx_algo, but same error code is generated.

Any help would be appreciated.

Thanks,

Abhishek

On Thursday, October 1, 2020 at 10:49:53 PM UTC+5:30 abhishek pandey wrote:

Hi, 

Is there any specific API or steps to decode the ALPHA channel from VP8 encoded Webm file. I've done the alpha encoding using webm_tools/alpha_encoder.

Thanks,

Abhishek

--
You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webm-discuss...@webmproject.org.

To view this discussion on the web visit https://groups.google.com/a/webmproject.org/d/msgid/webm-discuss/074844bc-1a74-4938-bc8c-95415a165ac0n%40webmproject.org.

[abhishek pandey]

I've guess the issue is in reading the frame additional data(side data mentioned above).

I'm using below code (mkvparser reference) to read the frame data.

const mkvparser::Block *mContainerBlock = NULL;

mkvparser::MkvReader *mContainerParser = NULL;

/*

****code for init****

*/

const mkvparser::Block::Frame &frame = mContainerBlock->GetFrame(mContainerBlockFrameIndex);

++mContainerBlockFrameIndex;
if (frame.len > static_cast<long>(*buffer_size)) {
delete[] *buffer;
*buffer = new uint8_t[frame.len];
if (*buffer == NULL) {
return -1;
}
}
*buffer_size = frame.len;
mContainerTimestamp = mContainerBlock->GetTime(mContainerCluster);

return frame.Read(mContainerParser, *buffer) ? -1 : 0;

Above code works for reading the frame data but I can see no option to read the additional block data(the alpha data), neither I found any reference for the same.

Reference : https://www.matroska.org/technical/codec_specs.html#block-additional-mapping

I guess I'm asking silly questions but I got no clue how to read the additional data from mkv format. 

Any help would be appreciated.

Thanks!

[James Zern]

On Mon, Oct 12, 2020 at 12:20 AM abhishek pandey <sup....@gmail.com> wrote:

I've guess the issue is in reading the frame additional data(side data mentioned above).

I'm using below code (mkvparser reference) to read the frame data.

const mkvparser::Block *mContainerBlock = NULL;

mkvparser::MkvReader *mContainerParser = NULL;

/*

****code for init****

*/

const mkvparser::Block::Frame &frame = mContainerBlock->GetFrame(mContainerBlockFrameIndex);

++mContainerBlockFrameIndex;
if (frame.len > static_cast<long>(*buffer_size)) {
delete[] *buffer;
*buffer = new uint8_t[frame.len];
if (*buffer == NULL) {
return -1;
}
}
*buffer_size = frame.len;
mContainerTimestamp = mContainerBlock->GetTime(mContainerCluster);

return frame.Read(mContainerParser, *buffer) ? -1 : 0;

Above code works for reading the frame data but I can see no option to read the additional block data(the alpha data), neither I found any reference for the same.

Reference : https://www.matroska.org/technical/codec_specs.html#block-additional-mapping

I guess I'm asking silly questions but I got no clue how to read the additional data from mkv format. 
Any help would be appreciated.

Not a silly question at all. Unfortunately BlockAdditional handling was never added to the legacy parser [1] as work was being done for a modernized version [2]. You might want to explore using webm_parser or ffmpeg's to retrieve this data.

[1] https://crbug.com/webm/1442

[2] https://chromium.googlesource.com/webm/libwebm/+/refs/heads/master/webm_parser/

[tee mo]

Have you resolved this issue? Can you play transparent webm normally? If it has been solved, please tell me how you did it, thank you