multi channel exrs

[Stefan Andersson]

Hello,

I searched the forum for multi channel exrs and saw that there is a limitation for multi channel exrs. Is this still the case or have I missed something?

best regards

stefan

--

Stefan Andersson - Associate VFX Supervisor - CG Supervisor

Important Looking Pirates - ILP

Email: stefan.a...@ilpvfx.com

Mobile: (+46) 073 626 8850

[John Haddon]

Hi Stefan,

EXR files can contain arbitrary numbers of channels, but they can also contain multiple parts, which is akin to multiple images within one file. I think I'm right in saying that the ImageReader can read both successfully, but that the ImageWriter will always write a single part with all the channels in that one part. Is this the limitation you are referring to?

Cheers...

John

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAPTgZjUj2CiE91QYt5jvxe948v6ph8ZzyDohoVWNg_S41yjc%3Dg%40mail.gmail.com.

[Stefan Andersson]

Hello!

I'm referring to getting aovs into a single exr.

regards

stefan

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVgJrj4ksbDqFdFqivuks7Qtz_qpPX4uSOi0FbyGOZzVGKw%40mail.gmail.com.

[Xuqiang Zhang]

merge AOV?

在 2020年5月29日星期五 UTC+8下午4:16:36，Stefan Andersson写道：

Hello!

I'm referring to getting aovs into a single exr.

regards

stefan

On Fri, 29 May 2020 at 09:50, John Haddon <jo...@gafferhq.org> wrote:

Hi Stefan,

EXR files can contain arbitrary numbers of channels, but they can also contain multiple parts, which is akin to multiple images within one file. I think I'm right in saying that the ImageReader can read both successfully, but that the ImageWriter will always write a single part with all the channels in that one part. Is this the limitation you are referring to?

Cheers...

John

On Fri, May 29, 2020 at 8:39 AM 'Stefan Andersson' via gaffer-dev <gaffe...@googlegroups.com> wrote:

Hello,

I searched the forum for multi channel exrs and saw that there is a limitation for multi channel exrs. Is this still the case or have I missed something?

best regards

stefan

--

Stefan Andersson - Associate VFX Supervisor - CG Supervisor

Important Looking Pirates - ILP

Email: stefan....@ilpvfx.com

Mobile: (+46) 073 626 8850

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAPTgZjUj2CiE91QYt5jvxe948v6ph8ZzyDohoVWNg_S41yjc%3Dg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVgJrj4ksbDqFdFqivuks7Qtz_qpPX4uSOi0FbyGOZzVGKw%40mail.gmail.com.

--

Stefan Andersson - Associate VFX Supervisor - CG Supervisor

Important Looking Pirates - ILP

Email: stefan....@ilpvfx.com

[Daniel Dresser]

Are you talking about Gaffer's image processing, or setting up Arnold outputs in Gaffer's Arnold binding?

As John says, there shouldn't be a problem with reading multiple channels in Gaffer's image processing stuff.

[Stefan Andersson]

Sorry, should have been more clear. It's when I create outputs for Arnold to render my AOV's (with the output node)

regards

stefan

On Fri, 29 May 2020 at 20:05, Daniel Dresser <dres...@gmail.com> wrote:

Are you talking about Gaffer's image processing, or setting up Arnold outputs in Gaffer's Arnold binding?

As John says, there shouldn't be a problem with reading multiple channels in Gaffer's image processing stuff.

--

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/83d4d085-ce32-4d3e-b4dd-d8b632e4d19e%40googlegroups.com.

--

Stefan Andersson - Associate VFX Supervisor - CG Supervisor

Important Looking Pirates - ILP

Email: stefan.a...@ilpvfx.com

[John Haddon]

I'm pretty sure we still have the same limitation there - one AOV per output file.

Cheers...

John

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAPTgZjW6LA%3D9NtP2%2Bc0gez0ayCqXjCBdjX02Tgv2W_CxQ5rTgA%40mail.gmail.com.

[Matthias Bjarnason]

Hey all,

Reviving a two year old thread here but has there been no update regarding this option for Arnold renders? This normally being a simple A simple check box in the aiAOVDriver - Maya, Houdini etc. OR some other workaround through Gaffer's output methods?

thanks,
- M

[rupert thorpe]

I am doing this kind of stuff to squash everything (slapcomp + layers + aov's) into one file after the render has written out:

I would agree that: being able to push more into each exr file coming from Arnold would simplify my life.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/2e256aab-787e-4078-9fbf-eb55e8a94761n%40googlegroups.com.

[EzequielM]

To expand on this, If it helps anyone there is an example box that gets the globals outputs, reads them with a CollectImages node and merges them

The box also does a post oidn denoise to individual aovs using a wedge.

https://github.com/ezequielmastrasso/LDTGaffer/blob/master/resources/boxes/LDTBatchRender.grf

This is used in the latest gaffer-examples repo

https://github.com/ezequielmastrasso/gaffer-examples

www.ezequielm.com

photos.ezequielm.com

Ezequiel Mastrasso

ezeq...@gmail.com

+353 83 0030034

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAK-bF_QRFupAQB0FwHz%3D%2B%2BJrNMgPrFQQa700O2uz%2BVHaH7%3DV7w%40mail.gmail.com.

[dan...@image-engine.com]

The merging approach that people are using works, but it's also true that we should support the Arnold options.  When you say that it's just a checkbox in aiAOVDriver, is that a toggle in the API you're talking about?  If it's a toggle in the API, you might be able to set it just by looking up the parameter name and setting it on an output.  But I think what's actually needed is that we need to recognize that multiple outputs use the same filename, and set them up to share the same driver.  This should be fairly straightforward, and we have laid some groundwork ( John has been playing with getting EXR layer names working properly in Arnold, and Gaffer 0.62 will add full support in Gaffer's image processing for reading/writing multipart EXR ), but I don't think we've done the Arnold part yet.

[Matthias Bjarnason]

Hey Daniel,

Honestly ... I don't find this as a toggle in the API (at a quick glance) ...but you're probably right - This might require a more elaborate setup to 'talk' with the driver(s). Looking forward to what updates 6.2 will bring.

Thanks guys (Ezequiel and Rupert) for the example files, though ... we'll check this out in the meantime, good to know there's at least some kind of workaround.

cheers!

[isabelle boeve]

Hi,

Reviving this  thread.

I've been trying to create .exr with multi channels for aovs for nuke.

Compositing prefers to use 1 .exr instead of multiple ones per aov.

Has there been any progress for this in Gaffer 1.2.1.1 ?

Thanks a lot in advance for the help!

Best,

Isabelle

[s.des...@gmail.com]

hello,

I also want to know if there are any update on this topic?

thx,

Seb

[John Haddon]

The situation hasn't changed I'm afraid :

- The ImageReader and ImageWriter nodes have full support for reading and writing multi-channel EXRs.

- The Arnold backend supports writing all light groups for a single AOV into a single EXR with the `layerPerLightGroup` parameter in the Outputs node.

- But we still don't support writing all Arnold outputs into a single EXR file.

We haven't prioritised any work in this area because the pipelines at the companies that employ us already have a post-render step where they denoise and combine the outputs to taste. You could do the same in Gaffer using CollectImages/ImageReader/ImageWriter as a task below the ArnoldRender node.

Cheers...

John

[isabelle boeve]

I did find a way by tweaking the ASCII .ass files generated from the gaffer scene

If you compare it with a .ass file from maya/arnold, where it is possible to render into 1 EXR, you'll find some similarities 

With some little tweaks you can turn the gaffer .ass output into a single multi channel EXR, when rendering

Works fine and the AOVs are technically correct.

I'll use this method until Gaffer has a better option.

Best,

Isabelle

[s.des...@gmail.com]

Hello,

I'm using Ezequiel Mastrasso's example to create a merge exr file but I have a problem with the aovs names in nuke. Instead of getting the original aovs name, I get either the aovs name followed by a suffix with the same name or a "main" suffix.

The question is, where do these suffixes come from? in the collect image, I can see the correct names of my aovs. Are the names modified when the exr is written?

I also have a problem with the cryptomatte which are split into 4 channels in the aovs list (crypto_material, crypto_material00, crypto_material01, crypto_material02). Why the crypto_matte is splitted ?

Do you know what I can do to fix these problems?

Thx,

Seb

[John Haddon]

What setting are you using for the `layout` plug on your ImageWriter? Gaffer provides a lot of flexibility in how layers are named and assigned to parts in the EXR file, and the default options follow the EXR spec to the best of our ability. There are also a bunch of options that emulate Nuke's behaviour, which as far as we understand it, deviates from the spec. There's some documentation for that here :

https://www.gafferhq.org/documentation/1.3.4.0/Reference/NodeReference/GafferImage/ImageWriter.html?highlight=imagewriter#layout

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/9def2d8c-974b-4693-bfd3-36eee338df10n%40googlegroups.com.

[Robert Kolbeins]

This is how my collect images,write images and output look like if that helps

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVgKKgMcVZezcVgE-9%3D%2BefDiGh%3D5%3Dn8hJ5BchQxK54kC41A%40mail.gmail.com.

[Robert Kolbeins]

This is one solution for the crypto, read them separately, merge and copy metadata

[Daniel Dresser]

The cryptomatte spec requires that cryptomatte is stored as several layers in the actual EXR file. Nuke probably shows a single aov if it recognizes the associated layers belong to the same Cryptomatte - definitely try out John's suggestion of using the `layout` plug on the ImageWriter to write Nuke-compatible names instead of EXR standard names - hopefully that gives you the result you expect ( I'm not actually familiar with whether Nuke is going to hide the multiple layers, I just no that they have to exist in the EXR file ).

Robert's suggestion to merge the multiple layers is simply discarding two thirds of the information in the Cryptomatte - the extra data is used to give correct edges when multiple objects overlap. Discarding that data will give you incorrect edges when you try to use the cryptomatte ( though if your scene doesn't have a lot of overlapping objects, you might not need to use full Cryptomatte data ).

[s.des...@gmail.com]

thank you for all this information. I'm gonna play with the layout settings and see if it works for me !!

Seb

[s.des...@gmail.com]

Hello,

I have now a working setup for most of my aovs.

I still have one issue with the naming of the cryptomatte and lightgroups aovs.

For instance :

crypto_object aovs should be

- crypto_object

- crypto_object00

- crypto_object01

- crypto_object02

but in my case it is :

- crypto_object_crypto_object

- crypto_object_crypto_object00

- crypto_object_crypto_object01

- crypto_object_crypto_object02

I guess there is a layer name and sublayer name added  to the aov name. I have tested different settings with the layout parameters in the imageWriter node but  I can't get rid off the crypto_object prefix.

Because of that aov name the cryptomatte in nuke doesn't recognize the crypto_object aov.

Do you know a way to remove it ?

Thx,

Seb

[John Haddon]

Hi Seb,

I don't have the answer to your question, but I do have a suggestion : use `exrheader` to show you the ground truth of what Gaffer is writing to the EXR file, rather than trust Nuke. In our experience, Nuke does not follow the EXR spec, so all is not necessarily as it appears. We ship `exrheader` with Gaffer so you can run it with `gaffer env exrheader myFile.exr` and it will tell you exactly what is in there. Hopefully that might give you some ideas about how to massage the data...

Cheers...

John

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/44aa032e-327a-43f6-a25d-4fb54a00e715n%40googlegroups.com.

[Robert Kolbeins]

Hi Seb

I had the same issue like you and only with arnold crypto outputs.  I couldn't get the collect images with merge metadata to work with the crypto.  My solution is to merge(over) the three crypto outputs as a one exr with copy metadata(image above).  That way when I load the exr in nuke using the crypto node I get the correct reading.  But as Daniel mentioned above there could be issues with edges which I need to double check if that the case.

-R  

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVg%2B--rqENVrU64Zv%3D5qdgnJrEwYWhc9x2LmZBX3jXai_SA%40mail.gmail.com.

[Sébastien Desmet]

It seems that it won't be easy to make it works... 

In nuke, I have tested to rename the channels with the use of shuffle and copy code and it works. I would like to do the same in gaffer but it seems there is no possibility to create a new channel with the shuffle node.

Is there a way to create a custom channel with the shuffle node ? or an other node ?

thx,

seb

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CANFqvZTeN7pPaC2c%3DkqAM80wSfdiYmEc%2B9kMcy6f5AFhYOg%2B-g%40mail.gmail.com.

[John Haddon]

Does that imply that the problem is that Gaffer is reading the image with the wrong channel names rather than the problem being in writing? If so, it might be worth looking at the `channelInterpretation` plug on the ImageReader node :

https://www.gafferhq.org/documentation/1.3.4.0/Reference/NodeReference/GafferImage/ImageReader.html?highlight=imagereader#channelinterpretation

This is another place where the EXR ecosystem seems to do stuff against the official spec, so by default we have to try to guess what was intended.

The Shuffle node has an ancient UI that should be brought up to standard to match the other shuffle nodes, but it is possible to shuffle into a new channel with it if you use the API to set it up :

```
root['Shuffle']["channels"][0]["in"].setValue( "test.R" )

root['Shuffle']["channels"][0]["out"].setValue( "newTest.R" )
```

Hope that helps...

Cheers...

John

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CA%2B0OZXX2B%3DKO1ZzwyVL%2ByA4k4hj6UW01dD1YGYWWLKx-rzWT2w%40mail.gmail.com.

[s.des...@gmail.com]

Thanks john,

I'm gonna test the shuffle script !

Here is also the EXR header that I have. I don't really know if something is wrong or not. ;)

I have tested my aovs in gaffer with the cryptomatte node and I have the same issue than nuke.

the non merge crypto_object aov is working and here is how it looks in the channel list of the catalog viewer :

RGBA

crypto_object

crypto_object00

crypto_object01

crypto_object02

they all have the a unique layer

when I try the crypto_object with the merge exr using the "part per layer" layout parameter, it doesn't work. Here is what I see in the channel list of the catalog viewer :

crypto_object :

           - crypto_object00

           - crypto_object01

           - crypto_object02

I have sub layer.

I have test different layout config but it doesn't really change

Thx,

Seb

[s.des...@gmail.com]

Hello,

I'm still playing with the merge exr and arnold denoiser (noice). I found a solution for the cryptomatte with the shuffle node thanks to your code snippet John. thx

I'm close to have something working but I still have one issue with the denoise of the lightgroups.

to make the arnold denoiser work, I have to write a "layerName" for each aov that I want to denoise. Otherwise the denoiser won't recognize the aovs.

the problem come from the lightgroup aov, as this aov is a merge exr with multiple layer, the "layerName" is not working as it doesn't know about all the layers inside the exr.

So I'm thinking about different solution to fix this :

-Is there a way to write a "layerName" by layer in a merge exr from the output node ?

here is how look my aov setting in the output node

- Is there a way to render the light group aov in separate files ? (one light group per file) (in mtoa you have the possibility to render all the light group in one exr or to render in different exr per lightgroup)

- Or do I have to build a light group aov per light my self ? (I would like to avoid this)

Thx,

Seb

[John Haddon]

Hi Seb,

Glad you're making progress.

-Is there a way to write a "layerName" by layer in a merge exr from the output node ?

I'm not sure I understand the question - can you phrase it very specifically in terms of what you need `exrheader` to report?

It might also be worth mentioning that we added the `layerPerLightGroup` checkbox to avoid the `_*` syntax for light groups. The main reason was so that we could support light groups for Light Path Expression AOVs, but it may also have some subtle relevance to the way layer names are handled. This is the way we'd expect it to be specified now :

Cheers...

John

[s.des...@gmail.com]

Hi John,

I have tested to set up the aovs like you did in the previous post but I still have problems.

Here is the setup: (I have added preserve_layer_name to see if it make a difference but not)

I have made a setup in maya and gaffer to be able to compare the exr metadata. Here is a link to the exr images : https://www.dropbox.com/scl/fo/qppdzi979a1etb4cornkd/h?rlkey=fv499t7m2gah1my5p8xk4bic7&dl=0

if you compare, you can see that the aovs name don't have the same structure. (I can shuffle the channels and rename them if needed)

But what is more problematic is that I don't have all the metadata in the exr that I need to be able to use the denoiser.

if we look at the maya render exr metadata, I can see this:

arnold/aovs/RGBA_left/filter (type string): "gaussian_filter"
arnold/aovs/RGBA_left/filter_width (type float): 2
arnold/aovs/RGBA_left/lpe (type int): 1
arnold/aovs/RGBA_left/source (type string): "RGBA_left"
arnold/aovs/RGBA_left_1/filter (type string): "variance_filter"
arnold/aovs/RGBA_left_1/filter_width (type float): 2
arnold/aovs/RGBA_left_1/lpe (type int): 1
arnold/aovs/RGBA_left_1/source (type string): "RGBA_left"
 

If I look at the gaffer render exr metadata, I can see this:

arnold/aovs/RGBA_left/filter (type string): "variance_filter"
arnold/aovs/RGBA_left/filter_width (type float): 2
arnold/aovs/RGBA_left/lpe (type int): 1
arnold/aovs/RGBA_left/source (type string): "RGBA_left"

As you can see I miss the RGBA_left aov.

I have the feeling that the layerName parametter is not working

Am I missing something ?

Do you have an idea why ?

Thx,

Seb

[John Haddon]

On Thu, Oct 26, 2023 at 5:24 PM s.des...@gmail.com <s.des...@gmail.com> wrote:

As you can see I miss the RGBA_left aov.

I have the feeling that the layerName parametter is not working

Sorry Seb, I'm still not quite understanding the problem - my fault I'm sure.

Thanks for providing the two files - much easier now we have something concrete to look it. Looking at them I see this :

• Both files have channels for `RGBA_<lightGroup>` for light groups "left" and "right".
• Maya alone has channels for light groups "left_1" and "right_1". What are those? Do you have light groups with those names?
• Gaffer also has channels for the "default" light group - that's something that Arnold makes automatically.
• Both have `arnold/aovs/*` metadata for any channels I mentioned above.

Oh, wait. Isn't the problem just that the Gaffer channel names are all prefixed with `light_groups`, so they no longer match the channel names referred to in the metadata? In that case, I think you should be able to control those channel names in Gaffer?

Let me know if I'm still getting the wrong end of the stick!

Cheers...

John

[s.des...@gmail.com]

Hi John,

I'll try to explain myself again, I may have misspoken :)
Here's what I want to do:
I'd like to do some merge exr from my arnold renderings. On top of that I would like to use arnold's denoiser (noice). As for the merge of exr, I've done it.
However, I can't get the denoiser to work properly. The lightgroups aovs are not recognised by the denoiser. I presume that the problem comes from the metadatas that the denoiser can't find in my exr.

As you can see in the metadatas of the maya render, I have the metadatas of the RGBA_left and RGBA_right aovs. To make the denoiser work, I have to add a 2nd output driver to my aovs RGBA_left and RGBA_right with a variance filter, which generates in the metadata of my exr, the aovs RGBA_left1 and RGBA_right1 (maya generates the name automatically).

When I try to do the same in gaffer, I can't get all these metadatas. In the rendering I've shared with you, for the aovs RGBA_left and RGBA_right, I can only see the metadatas linked to one type of filter (variance) but not the others.
maybe it has something to do with the name of the channels ?! 

Do you have any idea how I can fix this?

Thx,

Seb

[Murray Stevenson]

Hey Seb,

From the sounds of it, CollectImages’ behaviour of prefixing the collected images with a new layer name is causing the disconnect with the names in the `arnold/aovs/*` metadata and then Noice is unable to find the channels to denoise.

We’re proposing to add a plug to CollectImages to optionally disable this behaviour so CollectImages would just accumulate channels from the collected images without modifying each channel name. In this case it’d be up to you to ensure that each AOV being collected has unique channel names by enabling `preserve_layer_name` on the outputs and by specifying a unique `layerName` for each variance output. I’m assuming the missing metadata in your merged example image is because the lightgroup and lightgroup variance outputs end up sharing the same layer name so their metadata entries collide when merged.

I’ve prototyped this change in the attached script using a Loop setup to copy channels and metadata from each AOV into a single image. This preserves the channel names produced by Arnold, and if I write this image to a single-part EXR, then Noice appears to find the light groups and denoise them (according to the Arnold docs, Noice doesn't support multi-part EXR). The attached script shows an end-to-end task network producing an Arnold render, combining the AOVs and then denoising. I'm sure I'm missing some subtleties as I'm far from a Noice expert, but it's producing denoised light groups and AOVs out the other end... :)

This approach also keeps Cryptomatte happy, with both Gaffer’s Cryptomatte keyer node and Nuke’s accepting the combined image as the channel names and metadata match without any additional shuffling.

The CollectImages update isn’t a huge one so I expect we may be able to fit in for early next week, until then I think you could probably continue testing by adapting components from the attached script.

Cheers,

Murray

--

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/b47eebe4-8c9a-4326-b783-a98e99a29e48n%40googlegroups.com.

[John Haddon]

On Mon, Oct 30, 2023 at 8:43 PM Murray Stevenson <mur...@gafferhq.org> wrote:

We’re proposing to add a plug to CollectImages to optionally disable this behaviour so CollectImages would just accumulate channels from the collected images without modifying each channel name. In this case it’d be up to you to ensure that each AOV being collected has unique channel names by enabling `preserve_layer_name` on the outputs and by specifying a unique `layerName` for each variance output.

We've now implemented this and it's in code review. Shortly you should be able to download a build artifact from https://github.com/GafferHQ/gaffer/actions/runs/6735690043 if you'd like to test it out for yourself.

Cheers...

John

[s.des...@gmail.com]

Thank you very much guys !! I will test all of that !!

I'll let you know the progress

Thanks again for your time,

Seb

[John Haddon]

On Mon, Nov 6, 2023 at 10:01 AM s.des...@gmail.com <s.des...@gmail.com> wrote:

Thank you very much guys !! I will test all of that !!

 

Since I wrote my last message, we released the CollectImages improvements in 1.3.6.0, so you should be able to use the official build for your testing rather than the linked artifact.

Cheers...

John

[s.des...@gmail.com]

Hi John, Murray,

I have tested the latest release of Gaffer with the new collect image but I guess I miss something...

I can't see the lightGroups_noice in the exr metadata...

what are the settings that I have to use in the output to make it works ? I try with and without layerName but it doesn't make a difference

I pretty sure that I miss something obvious but I don't know what...

Thx for your patience ;)

Seb

[Murray Stevenson]

Hi Seb,

The output setup I’ve been using for testing is two light group outputs, both with `layerPerLightGroup` enabled, and with the “noice” output set to use the variance filter and a `layerName` of “RGBA_noice” (it appears the value of `layerName` is not super important as long as the name you choose is unique across all outputs, I've also had success in noice with `layerName` values of  "RGBA_1" and "GAFFERBOT"). A screenshot of those outputs is attached.

I’ve also updated my previous example to use the new CollectImages functionality from Gaffer 1.3.6.0. I think you could likely copy the “BatchOutputs” node from my example and try that in your script to see if you end up with all of the expected metadata after merging images.

Cheers,

Murray

[s.des...@gmail.com]

Hi Murray,

Thx for the example. I have tested your output in my script but no luck. So I render the example you provide and it didn't work neither.

Actually the command noice is working but it doesn't find the lightgroup_noice aovs during the denoise process.

If  check the exr metedata, I can see the RGBA_* aovs but there is no RGBA_*_noice aovs

Here is the metadata that I see in your example scene :

And this is the log that I see during the denoise :

Using feature AOV 'denoise_albedo' with filter 'gaussian_filter'
Using feature AOV 'N' with filter 'gaussian_filter'
Using feature AOV 'Z' with filter 'gaussian_filter'
Working with 1 frame at 640x480
Will denoise AOV "RGBA", using associated variance
   Output file will be "/uvfx/Homes/sdesmet/gaffer/projects/default/renders/arnoldLightGroupsAndNoiceCollect/merged_denoise.0001.exr"
   Could not find AOV with source RGBA_default
Could not find AOV "RGBA_default" to denoise, skipping.
   Could not find AOV with source RGBA_env
Could not find AOV "RGBA_env" to denoise, skipping.
   Could not find AOV with source RGBA_left
Could not find AOV "RGBA_left" to denoise, skipping.
   Could not find AOV with source RGBA_right
Could not find AOV "RGBA_right" to denoise, skipping.
Will denoise AOV "diffuse", using associated variance
   Output file will be "/uvfx/Homes/sdesmet/gaffer/projects/default/renders/arnoldLightGroupsAndNoiceCollect/merged_denoise.0001.exr"
Will denoise AOV "specular", using associated variance
   Output file will be "/uvfx/Homes/sdesmet/gaffer/projects/default/renders/arnoldLightGroupsAndNoiceCollect/merged_denoise.0001.exr"
Start denoising (patch radius 3, search radius 9, variance 0.5)
Denoising RGBA
Denoising diffuse
Denoising specular
Finished denoising

 

Thx again for your support !

Seb

[Murray Stevenson]

Hi Seb,

After taking another look at your example data, the render metadata suggests that you’re using Arnold 7.1.2.2 for the Gaffer tests but Arnold 7.2.4.1 for the Maya ones? If this is the case, then Arnold had a bug preventing the combined use of `layerPerLightGroup` with `layerName` that was only fixed in Arnold 7.1.3.0.

Gaffer prints a warning and prevents you from combining these features with an older Arnold version like 7.1.2.2, otherwise Arnold can crash. If you are running 7.1.2.2 then you may be seeing this warning at the very start of your renders:

`WARNING : ArnoldRenderer : Cannot use `layerName` with `layerPerLightGroup` for non-LPE output “Batch/Arnold/BeautyLightGroups_Noice”, due to Arnold bug #12282`

If you’re able to render with Arnold 7.1.3.0 or above, I suspect you may have more success. I’ve tried my example script with Arnold 7.1.2.2 and the lack of `layerName` support for light groups means that both the noice and regular light group metadata and channel names are set to RGBA_left/right/env/default and one set of channels & metadata then overwrites the other when the images are combined, and noice fails with a similar log to the one you've provided...

Cheers,

Murray

[s.des...@gmail.com]

Hi Murray,

Thx a lot, we will update Arnold and see if it works.

Seb

[s.des...@gmail.com]

Hi John, Murray,

With the update of Arnold, now everything is working perfectly !

Thx a lot for your help !!!

Seb

[John Haddon]

That's great! Thanks for letting us know.

--

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/51750168-2044-4022-8836-ce9c9d218816n%40googlegroups.com.

[Szabolcs Horvátth]

Hi,

This looks like an old thread but the multi layer EXR limitation still exists in 1.5.6.0 and solving the original problem at the core looks much simpler than post-merging the created exr files.

When exporting the scene to an ass file I saw that every output element creates a new driver_exr node (and filter node), regardless of the file path they use. This is something that causes problems in Arnold, there is even a warning about it when rendering the scene:
[aov] more than one output driver is associated to the file "%s"

What should happen (as in MtoA, HtoA, etc.) is that if multiple outputs point to the same path, than the Arnold options.outputs array should reuse the same driver node, and Arnold automatically saves all the AOVs to separate layers.

This could be done automatically by Gaffer, but if it sounds problematic for some reason, than you could expose a new string attribute for every Gaffer output array element to specify a driver node name as a custom attribute, and if multiple outputs share the same driver node than that should be reused for every one of them.

After exporting an ass file that created corrupt, unreadable EXR files because of the multiple drivers writing to the same EXR file, simply editing the outputs array of the file to point to the same exr node solved the problem:

From:

--

 outputs 3 1 STRING
  "albedo RGB ieCoreArnold:filter:Batch/Arnold/Albedo ieCoreArnold:display:Batch/Arnold/Albedo albedo"
  "diffuse_direct RGB ieCoreArnold:filter:Batch/Arnold/diffuse_direct ieCoreArnold:display:Batch/Arnold/diffuse_direct diffuse_direct"
  "RGBA RGBA ieCoreArnold:filter:Batch/Beauty ieCoreArnold:display:Batch/Beauty"

--

To:

--

  outputs 3 1 STRING
  "albedo RGB ieCoreArnold:filter:Batch/Arnold/Albedo ieCoreArnold:display:Batch/Arnold/Albedo albedo"
  "diffuse_direct RGB ieCoreArnold:filter:Batch/Arnold/diffuse_direct  ieCoreArnold:display:Batch/Arnold/Albedo diffuse_direct"
  "RGBA RGBA ieCoreArnold:filter:Batch/Beauty ieCoreArnold:display:Batch/Beauty"

--

So I'm pretty sure that if Gaffer simply exported unique driver nodes for all shared outputs than the problem would disappear.

Hope I did not misunderstood the original problem and stating the obvious.

Cheers,

Szabolcs

[Daniel Dresser]

Yeah, it is a relatively straightforward change. I've got a PR that implements this here: https://github.com/GafferHQ/gaffer/pull/6208

I think the only reason it hasn't been merged is that no one's flagged it as urgent to review. I guess this counts as a bit of a flag.

If you're doing a custom Gaffer build, you could grab that PR and confirm that it does what you need, but otherwise, hopefully it will get merged sometime soon.

-Daniel

[Szabolcs Horvátth]

Hi Daniel,

Thanks, its good to know that there is already a fix available! I haven't yet compiled Gaffer but perhaps this is a good reason to give it a try.

I don't say its a critical feature, but its an easy way to create broken image files, and that is something i would like to avoid exposing to users. 

Cheers,

Szabolcs

On Thu, Mar 6, 2025, 7:31 PM Daniel Dresser <dres...@gmail.com> wrote:

Yeah, it is a relatively straightforward change. I've got a PR that implements this here: https://github.comGafferHQ/gaffer/pull/6208

You received this message because you are subscribed to a topic in the Google Groups "gaffer-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gaffer-dev/kJQ5fuKIHP8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/61460a51-cba3-46b0-883f-5fdf0d14076en%40googlegroups.com.