Clarification on Color Management

[Austin Suszek]

Hi Folks,

I'm new to OpenFX (and frankly, new to proper color management too), and I wanted to make sure I understand how color is treated in OpenFX. 

I know there's an ongoing GitHub issue to add color metadata to the spec (is that still being worked on?) and I've found past conversations [1][2] that discuss color in this group. Based on those, this is what I've gathered for how it's currently implemented:

• Input and output images are always assumed to have a linear transfer function.
  • Side question: if that's true, is kOfxBitDepthByte ever used? A linear 8bit image would produce noticeable banding even before operating on it, right?
• If an image is 8bit or 16bit int, super-whites and sub-blacks are not representable. 
  • So any pipeline that wants to handle HDR content will require plugins to support float inputs/outputs?
• The assumed gamut for images is Rec. 709 (?)
  • I'm not too sure on this one, and I'm not even sure if it matters. I suppose many effects wouldn't care what the gamut is. One that might care would be a plugin that adds a solid border to an image. But even then, if I have that plugin applied and the host changes the working gamut from 709 to 2020, is the expectation that the border color would change or stay the same?
  • If it is important for all images to be Rec. 709, is the expectation that the host will map all WCG content to Rec. 709 and pass a float image with out of range values (more reason to require float plugins)?
  • If it's not important, I imagine a host would at least need generators to report the gamut they are outputting, otherwise a generator that decodes WCG content would always get misinterpreted. 

Let me know if I'm on the right track.

Thanks!

Austin

[Paul Miller]

• Input and output images are always assumed to have a linear transfer function.
  • Side question: if that's true, is kOfxBitDepthByte ever used? A linear 8bit image would produce noticeable banding even before operating on it, right?

I'm not sure you can assume a linear transfer function. I don't think the spec defines what the colorspace should be. kOfxBitDepthByte is supported by several hosts.

• If an image is 8bit or 16bit int, super-whites and sub-blacks are not representable. 
  • So any pipeline that wants to handle HDR content will require plugins to support float inputs/outputs?

As a plugin vendor you are only *required* to support float. What happens to it once it gets back to the host is up to the host - but most hosts support a full float workflow.

• The assumed gamut for images is Rec. 709 (?)

That would depend on the host. You can't assume that. In Nuke and Silhouette for instance, you can usually expect scene linear float. But you can force them both into sending through all sorts of colorspaces.

[Dennis...@sony.com]

In OpenFX you can’t assume or know host is using an particular color space, including gamut (Rec.709 vs. Rec.2020 vs. ACES) & tone curve (gamma vs. linear vs. log). That’s exactly why we wanted to add an extension for color space, but it keeps getting mired down in the details of general (gamma vs. log vs. linear) versus specific (S-Log3 vs. REDLog vs. etc.). The latest discussion included a suggestion that it includes a mandatory general description with fixed known tags and an optional more specific description for those apps that know more color spaces. There was also discussion of an extension to request particular color spaces, like you can say which bit depths you support.

 

I guess the one thing you can assume is that 8-bit is not linear <g>. At least it had better not be. But even moving to 16-bit you don’t know linear vs. not or where “white” is, and things get even more unpredictable in half/full float.

 

For our applications, most of our OpenFX processing is in our grading color space (e.g., Rec.709, Log, Rec.2020/S-Log3).

 

///d@

____________________________

Dennis Adams

Director of Technology

Sony Creative Software Inc

--
You received this message because you are subscribed to the Google Groups "ofx-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ofx-discussio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ofx-discussion/51a609a7-8c8b-4295-a7aa-615b671724c4%40fxtech.com.

[Austin Suszek]

Thanks for the quick reply!

I'm a bit confused though. I definitely understand the need for the color space extension, and I hope that will come soon and solve all these problems. But since it doesn't exist yet, plugins must be assuming something right? As a plugin developer today, how should I work with color to have the best chance (if not ensure) that I am applying an accurate effect?

Dennis, for your products, it sounds like there are multiple working color spaces to choose from. Do you pass images to OpenFX plugins as is (regardless of working color space), or do you perform some sort of conversion to keep the images consistent?

To view this discussion on the web visit https://groups.google.com/d/msgid/ofx-discussion/DM6PR13MB22650E47303500CE7BAE6823F1B29%40DM6PR13MB2265.namprd13.prod.outlook.com.

[Dennis...@sony.com]

For the few OpenFX plug-ins that really care about color space, they often have a parameter setting so the user can indicate what color space they think the image are in.

 

We don’t do conversion to keep images in some consistent color space for OpenFX. For the vast majority of our own plug-ins, color space doesn’t matter, since they just moving pixels around (sure, blending should technically be done in linear, but the video industry has been doing to “wrong” for decades for performance). Our color corrector works in the working color space, by definition.

 

Don’t get me wrong, we care deeply about color spaces, and all of our heavy-duty color management happens at the input side and output side, using a customized / extended OpenColorIO engine. If and when OpenFX understands color spaces, we’d use the same engine to handle conversions there too.

 

///d@

To view this discussion on the web visit https://groups.google.com/d/msgid/ofx-discussion/CACYGDtt8SWAu%3DiEeohv%3DauyHRUwReX2QLN24P3aqJdr16y9FtA%40mail.gmail.com.

[Austin Suszek]

Got it, it sounds like I'm just overestimated the importance of being "correct" with color.

I'll definitely use the color space extension when it comes out. But in the meantime I'll just act as if I'm receiving sRGB 8bit or linear rec709 16/32bit images, and not lose sleep if I'm actually being sent something different.

Thanks both of you for the help!

To view this discussion on the web visit https://groups.google.com/d/msgid/ofx-discussion/DM6PR13MB2265166635A5D35873460723F1B29%40DM6PR13MB2265.namprd13.prod.outlook.com.

[PierreJasmin_REVisionEffects]

As Dennis said, we when it's important for us have a menu to get us closer in ballpark internally (mostly for gamma 1.0 versus 2.2 here) and for apps we know we default the value to what makes most sense in that host.  Am all for the colo(u)r suite, but missing is an host to implement it (a condition for moving into release branch) so we can file FR at other hosts so they implement it too :)

Pierre

[Phil Barrett]

Hi Austin

Coming to this conversation a little late, sorry.

I write the OFX host code for Baselight, and as you'd imagine we're very
concerned about colour. Our colour space negotiation is shown at
https://github.com/ofxa/host-info/wiki/Baselight and something like this
might form the basis of any future standard.

For plugins that don't support our extensions, we too assume OFX images
need to be linear (because if you have to make a choice it's the only
sensible one) and we convert our images to and from linear when passing
them to and from plugins.

Phil
--
Phil Barrett
FilmLight

[PierreJasmin_REVisionEffects]

Phil, I have not looked in details at OCIO 2,  but it looks like they now have explicit transforms...  we might amend the suite a bit like I see USD did, with just minimal color spaces native and then OCIO.
USD did linear, rec 709 and OCIO as colorspaces...

[Dennis...@sony.com]

Supporting the idea of naming the transfer curve and primaries, I noticed that in the Khronos “Data Format” specification, they use transfer curve and primaries as a way of indicating color space.

https://www.khronos.org/registry/DataFormat/specs/1.3/dataformat.1.3.html#_color_space_composition (links to “12.1. Color space composition” section)

We could adopt similar in OpenFX. We just need to standardize what names to use and whether we make them separate or combines. I noticed Baselight uses different naming than we do; ours is "<curve>-<primaries>" for internal strings like "rec2020-slog3", "sgamut3.cine-slog3", "rec2020-hlg", and we show these as user-visible names like "Rec.2020/S-Log3", "S-Gamut3.Cine/S-Log3", and "Rec.2020/HLG" (respectively).

Sincerely,

///d@
____________________________
Dennis Adams
Director of Technology
Sony Creative Software Inc

-----Original Message-----
From: 'Phil Barrett' via ofx-discussion <ofx-dis...@googlegroups.com>
Sent: Friday, February 12, 2021 5:32 AM
To: ofx-dis...@googlegroups.com
Subject: Re: Clarification on Color Management

Hi Austin

Coming to this conversation a little late, sorry.

I write the OFX host code for Baselight, and as you'd imagine we're very concerned about colour. Our colour space negotiation is shown at https://urldefense.com/v3/__https://github.com/ofxa/host-info/wiki/Baselight__;!!JmoZiZGBv3RvKRSx!omPtqwAt3z7V33VDW1HhOfKTYEamNey8PZWu65rvhGUE82oEdhmg2UBYN4rkMRKl$ and something like this might form the basis of any future standard.

For plugins that don't support our extensions, we too assume OFX images need to be linear (because if you have to make a choice it's the only sensible one) and we convert our images to and from linear when passing them to and from plugins.

Phil
--
Phil Barrett
FilmLight

--
You received this message because you are subscribed to the Google Groups "ofx-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ofx-discussio...@googlegroups.com.

To view this discussion on the web visit https://urldefense.com/v3/__https://groups.google.com/d/msgid/ofx-discussion/367ac6d5f81b94623cbd4777a5b28cc7*40filmlight.ltd.uk__;JQ!!JmoZiZGBv3RvKRSx!omPtqwAt3z7V33VDW1HhOfKTYEamNey8PZWu65rvhGUE82oEdhmg2UBYNyTdYMuN$ .

[PierreJasmin_REVisionEffects]

Interesting, I complained about the lack of that on some standard managers, looks like they are doing it.
I am not the crack on this subject, whatever works for me :)

[PierreJasmin_REVisionEffects]

I just looked at the Khronos Data thing - although it's a good idea to avoid all these API that create a long enumeration of non-sense merging color space, bytes etc into a huge list , I don't see anyone ever supported that API unless I am wrong

Why can't we just use the syntax/model of OpenColorIo?    (basically just create a wrapper to associate OCIO to a buffer)
config (what the host supports) -  input colorspace - transform - output colorspace
for ui use name set in config + maybe family to group them under a single item

[Dennis...@sony.com]

AFAIK nobody uses that Khronos spec (it got pretty heavy from the original intent I think), but it had a nice section on color space that aligned with our discussion.

 

OpenColorIO is an API/SDK and doesn’t say anything about the naming of color spaces. There are some sample configs that show examples. Much more applicable is the current Academy ACES effort to product standard OCIO configs for ACES workflows, and they also have a color space naming convention we could pick up and borrow from.

 

In terms of transforms, that is the job of the host and/or plug-in, depending on what our color space extension looks like. Is the host just indicating color spaces to the plug, or does the plug-in have the ability to say what color spaces is supports (like OFX does for bit depths). If the former, it’s up to the plug-in to act on the color space indicated, perhaps converting it to something it prefers. If the latter, the host can do the transforms. Maybe OCIO gets used, but it should not be a requirement.

 

///d@

To view this discussion on the web visit https://groups.google.com/d/msgid/ofx-discussion/6d04859c-f0d5-4361-bbca-92048504b2f8n%40googlegroups.com.

[PierreJasmin_REVisionEffects]

Is this a suggestion to modify ofxColourSpace.h proposal for what is below?
Maybe what you suggest would work as a colorSpace parameter that you then populate as you want (and internally remaps to examples below???)

#define kOfxImageEffectPropColorHostManaged "OfxImageEffectPropColorHostManaged"

// Other defined ImageColourSpace can be added, adding and not supporting a ColourSpace is non-destructive

#define kOfxColourSpace_unknown_linear "OfxColourSpace_unknown_linear" // unknown primaries, linearised - this is the default in Baselight

#define kOfxColourSpace_unknown_unknown "OfxColourSpace__unknown_unknown" // - no conversion at all, no information at all

#define kOfxColourSpace_rec709_linear "OfxColourSpace__rec709_linear" // - Rec.709 primaries, linear tone curve

#define kOfxColourSpace_aces_linear "OfxColourSpace_aces_linear" // - ACES primaries, linear tone curve

#define kOfxColourSpace_arriwg_arrilogc "OfxColourSpace_arriwg_arrilogc" // - ARRI WideGamut primaries, ARRI LogC tone curve

#define kOfxColourSpace_sonysgamut3cine_sonyslog3 "OfxColourSpace_sonysgamut3cine_sonyslog3" // - Sony SGamut3.cine primaries, Sony SLog3 tone curve

#define kOfxColourSpace_rec709_rec1886 "OfxColourSpace_rec709_rec1886" // Rec.709 primaries, Rec.1886 tone curve(display - referred)

[PierreJasmin_REVisionEffects]

An additional note is a company like FilmConvert or something might be a better hand-shake than me as my interest is limited to make sure certain files *e.g. UV maps are not color transformed and also if the source is linear or not... but I do understand there are people handling more complicated "color" issues :)

Pierre