Alpha channel support
Sebastian Rosik
Alpha channel in webm.
Crazy? No. It would be great if i could use it in some games or
multimedia stuff (2d and 3d in webgl). Alpha channel in webm video
would be realy useful.
Yes, there is always canvas element - a bluebox effect isn't a big
deal, but for a designer with no coding skills, it could be a problem.
The performance of a video+alpha in canvas element could be very poor
compared to webm with 'native' support.
// xa
Andy Shaules
> --
> You received this message because you are subscribed to the Google Groups
> "WebM Discussion" group.
> To post to this group, send email to webm-d...@webmproject.org.
> To unsubscribe from this group, send email to
> webm-discuss...@webmproject.org.
> For more options, visit this group at
> http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
>
Franco Tecchia
sounds interesting. Could you explain your technique a bit more?
Franco
Sent from mobile.
Andy Shaules
All you need to do alpha is a single frame with which to do a difference
computaion as luma, applied to alpha. If the mask never changed, a single
alt ref frame could hold the data.
If the alt ref frame can be updated with compressed data on the fly, then
the mask too could be updated on the fly. I do a bit of composition in argb
color space, but it seems to me, most alpha cam is done with a blue or green
screen. In such a case, the single alt-ref frame can contain the blue screen
image without any actors.
As an alph channel decoder, it would be up to you to grab the alt-ref and
use it for dif->luma->alpha. Of course, what good is alpha if you cannot
composition with it, right?
So in the end, it appears to me it can already suppport an alpha layer, more
to the question is how or what'cha gonna' do wit' it?
Does the html5 video spec include support for alpha in a way that is useful
to an html page? Windowless video with actors walking across a text filled
div? Composting multi layers? Seems to me that anything useful could be made
with it as it is.
Andy
KenMcD
The talking presenter layered over a web page is very popular, and
growing.
But that is only one of the more visible uses for transparency.
The commercial VP6 codec with Flash Player is the only way to do this
now.
VP6 is the only codec which supports alpha-transparency that is
supported by any browser plug-in.
If WebM supported alpha-transparency users would finally have a free
open source unencumbered option for using transparent background video
in web pages.
That would be fantastic!
Gregory Maxwell
> Using alternate reference frame group, just set in your alpha mask.
Er. That would be quite a kludge— and would prevent you from using
that additional reference frame for its intended purpose. I don't see
why their would be any coding gain in combining it (it's not like the
alpha frame values will be useful predictors for the rest of the frame
or vice versa)... so why not just add another video track to the webm
stream with a second vp8 stream.
Some metadata would be needed to pair up that alpha channel, but
perhaps MKV already has something defined for that. It would probably
also be simpler than the metadata needed for an alt-ref splitting
approach— since that would also have to deal with fixing the timing.
Andy Shaules
entire video stream, in most cases you only need one single frame for the
entire duration of the video shot. Yes, you'd lose the alt ref frame for
decoding, and it would not be used for predictors. It would be used to
generate the alpha mask.
I'm not even saying it's a good idea, Im just saying its already possible as
is in a single track.
----- Original Message -----
From: "Gregory Maxwell" <gmax...@gmail.com>
To: <webm-d...@webmproject.org>
Sent: Tuesday, October 05, 2010 5:27 PM
Subject: Re: Alpha channel support
On Mon, Oct 4, 2010 at 3:05 PM, Andy Shaules <bowl...@gmail.com> wrote:
> Using alternate reference frame group, just set in your alpha mask.
Er. That would be quite a kludge� and would prevent you from using
that additional reference frame for its intended purpose. I don't see
why their would be any coding gain in combining it (it's not like the
alpha frame values will be useful predictors for the rest of the frame
or vice versa)... so why not just add another video track to the webm
stream with a second vp8 stream.
Some metadata would be needed to pair up that alpha channel, but
perhaps MKV already has something defined for that. It would probably
also be simpler than the metadata needed for an alt-ref splitting
approach� since that would also have to deal with fixing the timing.
Gregory Maxwell
> Yeah, it would be a kludge, but it would work. Rather than an adding an
> entire video stream, in most cases you only need one single frame for the
> entire duration of the video shot. Yes, you'd lose the alt ref frame for
> decoding, and it would not be used for predictors. It would be used to
> generate the alpha mask.
>
> I'm not even saying it's a good idea, Im just saying its already possible as
> is in a single track.
Enh. It still requires modifications to signal it and e.g. consider
what happens when the stream hits a keyframe.
flying sheep
> The talking presenter layered over a web page is very popular, and
> growing.
> But that is only one of the more visible uses for transparency.
>
> The commercial VP6 codec with Flash Player is the only way to do this
> now.
> VP6 is the only codec which supports alpha-transparency that is
> supported by any browser plug-in.
>
> If WebM supported alpha-transparency users would finally have a free
> open source unencumbered option for using transparent background video
> in web pages.
>
> That would be fantastic!
something which
1. Is useful in the web (Think of using controlless looping videos as
animated graphics)
2. Almost no other codec has.
Tomislav Horvat
problems within world of codecs. We all search for the best codec with
alpha but are stuck with a few codecs and their flaws.
On Oct 27, 12:50 am, flying sheep <trueflyingsh...@googlemail.com>
wrote:
KenMcD
After posting above I remembered that the only current codec
with alpha-transparency is the On2 VP6 codec.
So I am guessing that with the acquisition of On2
and the spin-out of the VP6 codec support
there may be some problems/restrictions about adding to VP8
the one feature which gives VP6 any competitive advantage.
So we may be barking at a tree.
Andy Shaules
not only add alpha, but stereo with alpha.
----- Original Message -----
From: "KenMcD" <kenmc...@gmail.com>
To: "WebM Discussion" <webm-d...@webmproject.org>
Sent: Saturday, November 20, 2010 7:47 AM
Subject: Re: Alpha channel support
>
John Luther
chatting about it informally w/ Chris Blizzard and Frank Galligan.
It's on my ToDo list to start a discussion thread here. Looks like
that's already been done, thanks, Sebastian!
Doing it in the codec itself would require changes to the VP8
bitstream, so yes, the most practical avenue would be to do it in
additional streams in the WebM container. Not very difficult, we just
need to bring together the smart people here to spec it out and agree
on the details.
--JL
On Nov 20, 11:09 am, "Andy Shaules" <bowljo...@gmail.com> wrote:
> I dont think theres any reason we can not add another channel or four and
> not only add alpha, but stereo with alpha.
>
>
>
>
>
>
>
> ----- Original Message -----
> From: "KenMcD" <kenmcd2...@gmail.com>
> To: "WebM Discussion" <webm-disc...@webmproject.org>
> Sent: Saturday, November 20, 2010 7:47 AM
> Subject: Re: Alpha channel support
>
> > After posting above I remembered that the only current codec
> > with alpha-transparency is the On2 VP6 codec.
>
> > So I am guessing that with the acquisition of On2
> > and the spin-out of the VP6 codec support
> > there may be some problems/restrictions about adding to VP8
> > the one feature which gives VP6 any competitive advantage.
>
> > So we may be barking at a tree.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "WebM Discussion" group.
Silvia Pfeiffer
the same time? Is there even anything close to a "standard" for stereo
video? I'm wondering because of all the hype around stereo movies and
stereo TVs at the moment.
Silvia.
> To post to this group, send email to webm-d...@webmproject.org.
> To unsubscribe from this group, send email to webm-discuss...@webmproject.org.
Franco Tecchia
mean to encode efficiently not just a stereo pair (that is very limiting in
terms of playback) but an additional channel containing the Z-buffer for
each pixel in the Y channel of the image (similarly to Microsoft Kinect).
Please note that usually such z-channel has a 16 bits depth.
A good support for RGBA+Z would pave the ground for many great applications.
Franco
Steve Lhomme
to WebM. There are already test builds of FireFox 4 that works with
YouTube 3D samples (I tried it successfully this week-end). For now
it's using side-by-side video and with each side shrinked to half
resolution. That's not necessary, but that makes it easier to tune the
decoders, they just have to be able to decode the same as 2D picture.
No sync issues either if 2 different streams were used. So it's likely
to be the preferred way for a while.
Steve
--
Steve Lhomme
Matroska association Charmain
John Luther
having working prototype by the end of Q1.
Steve Lhomme and I were discussing offline last week and he has some
implementation ideas. Steve, can you summarize?
Thanks--
JL
--
John Luther
Product Manager, WebM Project
On Nov 22 2010, 5:47 am, Steve Lhomme <slho...@matroska.org> wrote:
> Streoscopic video is possible in Matroska and has been added recently
> to WebM. There are already test builds of FireFox 4 that works with
> YouTube 3D samples (I tried it successfully this week-end). For now
> it's using side-by-side video and with each side shrinked to half
> resolution. That's not necessary, but that makes it easier to tune the
> decoders, they just have to be able to decode the same as 2D picture.
> No sync issues either if 2 different streams were used. So it's likely
> to be the preferred way for a while.
>
> Steve
>
> On Mon, Nov 22, 2010 at 12:58 AM, Silvia Pfeiffer
>
>
>
>
>
>
>
>
>
> <silviapfeiff...@gmail.com> wrote:
> > As it has been mentioned: how difficult would it be to do stereo at
> > the same time? Is there even anything close to a "standard" for stereo
> > video? I'm wondering because of all the hype around stereo movies and
> > stereo TVs at the moment.
>
> > Silvia.
>
> >> How timely!Alphawould be a very cool feature in WebM & I've been
> >> It's on my ToDo list to start a discussion thread here. Looks like
> >> that's already been done, thanks, Sebastian!
>
> >> Doing it in the codec itself would require changes to the VP8
> >> bitstream, so yes, the most practical avenue would be to do it in
> >> additional streams in the WebM container. Not very difficult, we just
> >> need to bring together the smart people here to spec it out and agree
> >> on the details.
>
> >> --JL
>
> >> On Nov 20, 11:09 am, "Andy Shaules" <bowljo...@gmail.com> wrote:
> >>> I dont think theres any reason we can not add another channel or four and
> >>> not only addalpha, but stereo withalpha.
>
> >>> ----- Original Message -----
> >>> From: "KenMcD" <kenmcd2...@gmail.com>
> >>> To: "WebM Discussion" <webm-disc...@webmproject.org>
> >>> Sent: Saturday, November 20, 2010 7:47 AM
> >>> Subject: Re:Alphachannel support
>
> >>> > After posting above I remembered that the only current codec
> >>> > So I am guessing that with the acquisition of On2
> >>> > and the spin-out of the VP6 codec support
> >>> > there may be some problems/restrictions about adding to VP8
> >>> > the one feature which gives VP6 any competitive advantage.
>
> >>> > So we may be barking at a tree.
>
> >>> > --
> >>> > You received this message because you are subscribed to the Google Groups
> >>> > "WebM Discussion" group.
> >>> > To post to this group, send email to webm-disc...@webmproject.org.
> >>> > To unsubscribe from this group, send email to
> >>> > webm-discuss+unsubscr...@webmproject.org.
> >>> > For more options, visit this group at
> >>> >http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
> >> To post to this group, send email to webm-disc...@webmproject.org.
> >> To unsubscribe from this group, send email to webm-discuss+unsubscr...@webmproject.org.
> > --
> > You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
> > To post to this group, send email to webm-disc...@webmproject.org.
> > To unsubscribe from this group, send email to webm-discuss+unsubscr...@webmproject.org.
Steve Lhomme
There are 2 main ways to handle 3D. It's actually a lot like adding 3D
support to existing 2D streams. They are all container based as VP8 is
not going to be changed for that. So for each video frame there will
be another frame (likely greyscale) representing the alpha values to
apply to each pixel. So the dimension for that frame should match the
one of the "original" stream.
The 3 ways to handle that in the header is as follows:
1/ BlockAdditions: as the name suggests is additional data tied to the
Block. These data have an ID to know what they represent (1 would mean
alpha channel with VP8). As it is tied to the Block it has the same
timestamp. Existing player with no knowledge of it just discard that
info. The main different with existing WebM muxers is that they have
to support BlockGroup/Block in addition to SimpleBlock with was
preferred so far.
There is also a field in the Track Entry to specify that
BlockAdditions is used. So that the player/web browser can know an
alpha channel is in there.
http://www.matroska.org/technical/specs/index.html#BlockAdditions
2/ TrackOperation: is used to combien multiple tracks together. To
create a 3D effect for example. In this case the Blocks for the
regular and alpha are separated in different tracks and combined on
playback. It has the advantage of working with any codec combination.
Right now only "PlaneCombine" and "JoinBlocks" operation are possible,
but a "AlphaChannel" one could easily be added.
http://www.matroska.org/technical/specs/index.html#TrackOperation
Solution #1 has the advantage that the data are tied to the original
codec (VP8 here) and so the greyscale part could make use of other
compression information available in the source frame. That should
lead to better compression of the alpha channel. That means the VP8
API might need some changes to allow handling this as well.
I suppose on playback both solutions are equivalent.
Steve
> To post to this group, send email to webm-d...@webmproject.org.
> To unsubscribe from this group, send email to webm-discuss...@webmproject.org.
> For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.
>
>
--
James Cloos
SL> So for each video frame there will be another frame (likely greyscale)
SL> representing the alpha values to apply to each pixel. So the dimension
SL> for that frame should match the one of the "original" stream.
Is it a strict requirement that the extra frame exactly match the pixel
dimensions of the main frame? It could be useful to have an alpha pixel
map to multiple original pixels, akin to how 4:2:2, 4:1:1 and 4:2:0 map
colour difference channels to the Y channel.
-JimC
--
James Cloos <cl...@jhcloos.com> OpenPGP: 1024D/ED7DAEA6
Vladimir Pantelic
> Yes,
>
> There are 2 main ways to handle 3D. It's actually a lot like adding 3D
> support to existing 2D streams. They are all container based as VP8 is
> not going to be changed for that. So for each video frame there will
> be another frame (likely greyscale) representing the alpha values to
> apply to each pixel. So the dimension for that frame should match the
> one of the "original" stream.
>
> The 3 ways to handle that in the header is as follows:
> 1/ BlockAdditions: as the name suggests is additional data tied to the
> Block. These data have an ID to know what they represent (1 would mean
> alpha channel with VP8). As it is tied to the Block it has the same
> timestamp. Existing player with no knowledge of it just discard that
> info. The main different with existing WebM muxers is that they have
> to support BlockGroup/Block in addition to SimpleBlock with was
> preferred so far.
this is assuming the webm spec is less frozen than the VP8 one.
> 2/ TrackOperation: is used to combien multiple tracks together. To
> create a 3D effect for example. In this case the Blocks for the
> regular and alpha are separated in different tracks and combined on
> playback. It has the advantage of working with any codec combination.
> Right now only "PlaneCombine" and "JoinBlocks" operation are possible,
> but a "AlphaChannel" one could easily be added.
>
> http://www.matroska.org/technical/specs/index.html#TrackOperation
does webm support that per the spec today?
does google have plans for VP8/webm and 3d already? If yes, re-using
that for alpha might make sense...
Steve Lhomme
BlockAdditions data to produce the alpha channel. In that case any
kind of internal geometry/pixel coding is possible.
On the output side, I don't know what is best for browsers. If they
would like YUV+8 bit greyscale or ARGB pixels or even a completely
different coding.
Steve Lhomme
> Steve Lhomme wrote:
>>
>> Yes,
>>
>> There are 2 main ways to handle 3D. It's actually a lot like adding 3D
>> support to existing 2D streams. They are all container based as VP8 is
>> not going to be changed for that. So for each video frame there will
>> be another frame (likely greyscale) representing the alpha values to
>> apply to each pixel. So the dimension for that frame should match the
>> one of the "original" stream.
>>
>> The 3 ways to handle that in the header is as follows:
>> 1/ BlockAdditions: as the name suggests is additional data tied to the
>> Block. These data have an ID to know what they represent (1 would mean
>> alpha channel with VP8). As it is tied to the Block it has the same
>> timestamp. Existing player with no knowledge of it just discard that
>> info. The main different with existing WebM muxers is that they have
>> to support BlockGroup/Block in addition to SimpleBlock with was
>> preferred so far.
>
> this is assuming the webm spec is less frozen than the VP8 one.
The way Matroska works, you can add fields on be fully backward
compatible. Non existing fields are just discarded. In that respect
(and from what I know all WebM parsers respect that) both solutions
will be backward compatible. Just think of it as an XML file with
added fields/entities.
>> 2/ TrackOperation: is used to combien multiple tracks together. To
>> create a 3D effect for example. In this case the Blocks for the
>> regular and alpha are separated in different tracks and combined on
>> playback. It has the advantage of working with any codec combination.
>> Right now only "PlaneCombine" and "JoinBlocks" operation are possible,
>> but a "AlphaChannel" one could easily be added.
>>
>> http://www.matroska.org/technical/specs/index.html#TrackOperation
>
> does webm support that per the spec today?
I'm not sure the StereoMode field has been formally added to the WebM
variant of the specs. But there are 3D WebM files on the web that work
in nightly builds of Mozilla with nVidia 3D glasses. So 3D WebM is
already a reality.
> does google have plans for VP8/webm and 3d already? If yes, re-using
> that for alpha might make sense...
--
Frank Galligan
Frank