Texture lookup and Alpha

[Master Zap]

So.... it seems doing a texture lookup with Alpha in OSL, the lookup function already multiplies in the alpha into the RGB values. 

• Am I right, or was I smoking something?
  
• Is this intentional?
  
• If so, why?
  

I'm personally kinda expecting to get the values that is in the file, with no processing, when I do a lookup w. alpha?

/Z

[Larry Gritz]

What kind of file, and are you really dealing with unassociated alpha in texture maps?

--

Larry Gritz

l...@larrygritz.com

[Master Zap]

a) PNG
b) Yes :) ?

I may have my reasons to treat texture data as.... well... data?

/Z 

[Larry Gritz]

This is actually a feature of the texture system.

When you set up the renderer, do this:

    shadingsys->texturesys()->attribute ("unassociatedalpha", 1);

That will cause textures to NOT be automatically turned into 'associated alpha' as they are read in. It is then totally up to you to figure out what to do with those values in your shader. You can no longer count on all texture lookups being "associated" (i.e. "premultiplied") regardless of what kind of file they come from.

--

Larry Gritz

l...@larrygritz.com

[Master Zap]

Ah okay. Good to know!

It's a little brute force it's part of the entire texture system, though, I wouldn't mind if it was a per-texture setting. Some textures hold "data" more than they hold pixel values, others don't.

Anyway it explains (and motivates) the behavior; so now I know. Thanks.

/Z

[Larry Gritz]

How would you want this to work? Is this something that's inherent in the file itself? Predeclared by the renderer? In the shader itself when it calls texture?

--
You received this message because you are subscribed to the Google Groups "OSL Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osl-dev+u...@googlegroups.com.
To post to this group, send email to osl...@googlegroups.com.
Visit this group at https://groups.google.com/group/osl-dev.
For more options, visit https://groups.google.com/d/optout.

--

Larry Gritz

l...@larrygritz.com

[Master Zap]

Good question. My thinking was to do it in the texture lookup stage, but maybe alpha interpretation affects how mip-levels are generated (Does it? I dont' remember) ?

But an ability to declare this on the texture level somehow would work too. (Though "somehow" needs to be defined, then :) )

To be clear, this is not a pressing need or even a requirement I have just now; I just noticed the behaviour, wondered about it, and got my answer.

No particular action required at this point.

/Z

[Master Zap]

Follow up question on this: Is the alpha behavior file-format dependent??

Seems PNG gives you premultiplied alpha but .exr does not?

And if it is format- dependent, how do I know?

/Z

[Larry Gritz]

Well, the situation generally is pretty awful.

As it turns out, the PNG file format specification dictates that color should not be premultiplied (also known as "unassociated alpha"), and the OpenEXR specification is pretty clear that color should always be premultipled (also known as "associated alpha"). Some other file formats allow either choice, for example TIFF has a metadata field that's supposed to say if the "extra" channel (beyond RGB) is supposed to be associated alpha, unassociated alpha, or neither/don't know.

Short answer:

By default, OIIO's texture system will automatically premultiply upon input, for image files that appear to be unassociated, so from OSL's point of view, you should treat all texture inputs as premultiplied.

There is an option (as discussed earlier in this thread) when the renderer to initialize the texture system that will turn off this behavior, i.e., to leave unpremultiplied colors as they were in the file. If you do that, then, the shader itself needs to do the premultiplication to use it as ordinary texture (though. You can probably *mostly* tell by looking for file metadata "oiio:UnassociatedAlpha" is present and nonzero. So you could imagine shader code that looks like this ugly mess:

    color Ctex;

    float Atex;

    Ctex = texture (filename, s, t, "alpha", Atex, "fill", 1.0);

    int unassociated = 0;

    gettextureinfo (filename, "oiio:UnassociatedAlpha", unassociated);

    if (unassociated)

        Ctex *= Atex;

The good news is that the gettextureinfo will probably constant-fold away, and so will the multiply if it's not an unassociated file, that is, if the runtime optimizer can determine the name of the file used by the shader when it's being JITed, so this is not nearly as expensive at runtime as it appears. The bad news, though, is that it's not 100% clear to me that the results of texture-filtering-then-premultiplying are correct compared to premultply-then-filter. Other bad news is that you are relying on OIIO input reader for that file, which is probably fine for exr or png (where it's supposed to always be one way), but like I said for TIFF also requires trust that it was correct in the metadata in the first place. If the metadata is wrong, you're probably hosed either way.

Long answer, including background info and rant:

At this point, I'm going to go on a bit of a religious rant and just say that associated alpha (premultiplied color) is the only choice that makes sense. It's the only thing that renderers are likely to produce naturally, it's the only way that compositing and any other image processing math works, and in my opinion it's the only sensible form to store any image. It is also worth noting that associated alpha can express RGBA values that unassociated alpha (un-premultipled color) CANNOT. It's just a fact of life; for example, an RGBA pixel that means "glow bright yellow but don't block the view of the things behind it" can only be expressed in premultiplied colors (associated alpha).

Except... except that unassociated alpha (which is equivalent to a solid color and a mask) was a familiar analogy to painters and graphic designers, and saved you some precision problems if you were stuck with 8 bits per channel in the old days (and assuming you didn't need any of those transparent but glowy pixels), and the math didn't seem too bad if you were developing your application prior to publication of Porter & Duff's seminal 1984 paper "Compositing Digital Images" (or indeed after that, if you were somehow unaware of the paper or didn't understand it). So we have a lot of apps (Photoshop, I'm looking at you) and file formats (PNG, etc.) whose architectures are geared toward the inferior way to represent pixels. So the use of unassociated/unpremultiplied files continues in some circles.

OK, rant over. Thanks for sticking with me. Back to the main topic...

So the way that OpenImageIO (which supplies the texture system used by most renderers using OSL, and they heavily influence each other) deals with this is to attempt to determine (through metadata or file type) its best guess on whether an image being read is associated or unassociated alpha, and for the latter will automatically premultiply in order to present the pixels on the app side of the API as if they were premultiplied.

Which means that, I think, in OSL you ought to be able to just carry on as if all colors were premultiplied.

* Best practice, IMHO: Write OSL as if all texture inputs are premultiplied, and let OIIO's texture system do that for you automatically.

But every once in a while, somebody wants the original pixels, in order to ensure that you don't lose color precision in the premultiplication (this is really only important for 8 bit per channel images). There is a way for the renderer to tell the texture system, as described earlier. But as I said, this leaves you with having to deal with both cases in the shader.

I'm open to better suggestions for how to deal with this mess.

[Master Zap]

Thanks Larry. I can feel your rant..... 

Ah, yes, Adobe and Alpha and EXR.....

I have an AWESOME anecdote about that. I will tell you some day over beer.

Or I can post it here, if you really want me to :)

/Z

[Michael Wolf]

Hallo Zap,

On Monday, October 22, 2018, 4:04:20 PM, you wrote:

Thanks Larry. I can feel your rant..... Ah, yes, Adobe and Alpha and EXR..... I have an AWESOME anecdote about that. I will tell you some day over beer.

So it's not the one that we all know and followed on the OpenEXR ml? ;)

Cheers,
Mike

--
db&w Bornemann und Wolf GbR
Seyfferstr. 34
70197 Stuttgart
Deutschland
 
michae...@db-w.com
 
http://www.db-w.com
 
tel: +49 (711) 664 525-3
fax: +49 (711) 664 525-1
mob: +49 (173) 66 37 652
 
skype: lupus_lux
msn:   michae...@db-w.com

[Larry Gritz]

I suspect I already know that anecdote (how many Adobe + EXR +alpha anecdotes can there be?). No need to embarrass the clueless parties here.

--
Larry Gritz
l...@larrygritz.com

[Master Zap]

Lol yes that's the one. I guess you were all there... :)

/Z

[CrestChristopher]

Could I know ?

Christopher