H.264 Alpha Channel

0 views

Skip to first unread message

Karleen Chura

unread,

Aug 3, 2024, 4:32:02 PM8/3/24

to forbovocer

The H.264 standard describes alpha channel encoding through auxiliary picture. But I have never seen a decoder or encoder supporting it.So to my knowledge - yes there is standard for H.264 with alpha but to me there is no known implementation.So you would have to implement or modify existing H.264 decoder/encoder which is a substantial task.

I think the situation is even worse - I don't know of any popular cross platform video format with alpha. On Apple I'd use Apple ProRes 4444 - but you may have a hard time creating ProRes files on anything but macOS.

Ok, there's some codecs/formats that can export transparency but most of them are intermediate formats (set for best quality not small size) (small list of those you can see here: List of video formats supporting alpha channels) so it will give you bigger file size but if that is not an issue you can use any of those. If you are on windows you can also check Lagarith - lossless avi codec that gives you transparency. If any of those formats/codecs doesn't satisfy your needs then check as I've sai

h264 is unnable to export aplha channel (transparency) so you have to use something else to render it. I'm not familiar with Twitch channel settings but mae there is some other file format that can give you alpha channel f.eg. swf

I've already said .eg. SWF. There also other formats but not all are suitable for something like overlays on live footage so it would be much easier if you could tell us what video/animation formats are supported by Twitch/broadcast system.

The overlay is actually a video and I don't know how to get transparency or alpha in adobe media encoder but I was able to render the video with alpha in after effects using avi which made a 6 GB file and Quicktime which made a 700 MB file.

Basically I use a program called OBS which I connect to my twitch and use a plugin to play videos. With the video plugin if I have a transparent video then I can have an overlay animate on with monitor capture or game capture behind the video.

You can render your RGB and alpha (matte) channel separate from each other, that way you can get away with pretty small file sizes. H264 does not support integrated alpha channels, even prores only does that in 4444 quality.

For Twitch stream overlays, I recommend the WebM container since it provides RGB+A. You can download the codec here for use in Adobe Media Encoder (AME). Install it and it will appear in your list of encoding options in AME. Just remember to check "Include Alpha Channel".

What an odd thing: I've been scouring the internet recently trying to find modern video formats that support alpha channels (aka "transparency layers"). Almost every forum I've reviewed references an article that has not been updated since 2008. Since it is now 2021 and over a decade later, I figured I would try to provide an updated version. This should be especially useful for Twitch streamers (stream overlays) and OBS/Streamlabs users.

Alpha channels are essentially layers of transparency on an image or video. They're color components representing the degree of transparency (or opacity) of a color (i.e., the red, green and blue channels). They are used to determine how specific pixels are rendered when blended with other pixels. Generally, alpha channels are represented by a white and gray checkerboard or a solid black background.

So. Many. Things. They're used for compositing images and footage, making memes and product cutouts, or in my case - displaying a video on the web on top of a colored background (this is an issue that deserves its own blog post).

There are two ways of depicting an alpha channel. Straight (also called "unmatted," or "unassociated") and premultiplied ("matted," or "associated"). These only affect the color channels, and actually aren't related to the alpha channel itself.

Straight alpha channels can only be loaded in applications that support it, otherwise, it'll appear as a black screen. However, they retain more accurate color information than pre-multiplied alpha channels do. Straight channels retain more accurate color information than premultiplied channels. This is because it separates the RGB color values from the alpha values and stores them independently of each other - hence the term "unassociated". It doesn't matter what you place the image on, it'll look the same. You'll want to use straight alpha channels 98% of the time.

Premultiplied alpha channels work a bit differently. It mixes the alpha channels with the RGB color values ahead of time upon export. The upside to this is the image or video can be viewed agnostically on almost anything. However, it has a big negative side effect: since the image's alpha is already determined upon export, it'll take whatever the background WAS in the program you used to create it and mix it with the image, leaving a halo-like edge around your cut-out. This happens because the colors of semitransparent areas are shifted toward the background color in proportion to their degree of transparency, aka "matted" onto the image. Certain programs like After Effects have the ability to "unmult" layers, removing this effect for you.

The short answer is that h.264 does not support an alpha channel since no decoder/encoder support the functionality. See this creativecow forum post for more info. While an alpha channel does not work in the h.264 video itself, it is possible to use 2 different h.264 videos to get the same effect. See my post to the creativecow forum for more info.

(PDF download only) _pub.asp?lang=e&id=T-REC-H.264-201106-S!!PDF-E&type=itemsyou can read about it here. It's a bit dense, but the International Telecommunication Union's spec for H.264 encoding explicitly states transparency(and additional luma maps) as being a use for the specification's auxiliary channel. However it also notes "The... decoding of auxiliary coded pictures is not required" and I don't think it was ever used in an official capacity.

The confusion is likely because mp4 and quicktime can both use the h.264 codec, but quicktime, (both the container and player) supports transparency, so theoretically mp4 can be encoded with it as well, though it would require a dedicated decoder. I have seen a hack that simply encodes an alpha layer to another video or at the end of the file. Adobe video export can default to uncompressed RGBA in the case that quicktime alpha output is requested, meaning you can ask for h.264 quicktime, request alpha, and get a quicktime with transparency that is not h.264. This confused me in testing and took a couple tries to figure out what had actually happened to my file. Combined with confusion over quicktime player and wrapper(they support both H.264 and alpha but not simultaneously), all of this could be a source of public confusion.

This is incredibly late, I understand, but these videos, I believe, primarily come from After Effects' and Adobe's encoders and decoders, which are Apple-blessed, and therefore take advantage of Apple quicktime player's RGB+A decoding update which was not related to its h.264 characteristics. Adobe removed them when Apple deprecated both quicktime and the de4dicated player, but the wrapper was added back after years of public protest because there was no functionality to replace the workflow. It is indeed a saga.

Apple has deprecated quicktime, and it's been deprecated for years, but is still in active use in VFX because to my knowledge neither Apple nor Adobe supplied a replacement for video that was both heavily compressed and supported alpha. Webm is looking like a modern replacement, but its support by third parties is spotty, and AV1, touted as an h.264 replacement does not seem to officially support it at all.

(edited for correctness- mostly the bold stuff. sorry people who noted I had it a little backwards! quicktime being deprecated and prores and uncompressed RGBA using the same extension as quicktime really obfuscates things... premiere had the codec ommitted because it was uncompressed, which turned me around as I hadn't requested it.)

Many file formats can include an alpha channel, including Adobe Photoshop, ElectricImage, FLV, TGA, TIFF, EPS, PDF, and Adobe Illustrator. AVI and QuickTime (saved at a bit depth of Millions Of Colors+), also can contain alpha channels, depending upon the codec (encoder) used to generate the images stored in these containers. For Adobe Illustrator EPS and PDF files, After Effects automatically converts empty areas to an alpha channel.

This blog post shows how it can be done. A few things though. h.264 is a delivery codec, it should not be used as an intermediate in the video production process. Export from AE using the Quicktime Animation codec at 32 BPP (Millions+) to retain the alpha channel. Dave is correct that h.264 does not support an alpha channel by default, one needs to actually create 2 videos, one holds the RGB and the other holds the alpha pixels as a grayscale video. All the code to split the videos, convert to h.264 via ffmpeg+x264 and then recombine the videos on the iPad are already included with the AVAnimator tools. I think your will be quite impressed with the results as the generated file sizes are quite small and the .m4v files generated can be included directly to your xcode project.

QLab is finicky with h264 videos. Below is from Figure53 recomedations on video codecs.V/r
Bennamin
Video cues require a target, which must be a file containing video. While QLab can play files in any format supported by AVFoundation, we recommend the following formats, listed in order of preference, for videos without transparency:ProRes 422 Proxy
ProRes 422 LT
PhotoJPG
H.264
For videos with transparency, also referred to as alpha channel support, AVFoundation only supports ProRes 4444.