Custom Gaffer 3Delight updates

[Goran Kocov]

Hi everyone!

First a huge THANK YOU! to Eric and John for the amazing recent work on adding significant functionality to Gaffer’s 3Delight support and to Aghiles and the 3Delight team for making an awesome renderer with an excellent API that is at the same time so easy to work with and so flexible.

After tinkering with adding some of the missing 3Delight features by post-processing and then rendering the Gaffer exported NSI scene with a batch of Python Command and System Command nodes (which worked surprisingly well, mostly due to the amazing power and flexibility that Python Command, System Command and the general Gaffer Dispatch structure provides, but was limited to simpler scenes), I decided to take a deep dive and try implementing the features directly in Gaffer.

Working with and compiling Gaffer’s source code turned out to be much simpler than I feared it would be (once again thanks John!) and I’m happy to present you a Linux build of Gaffer that has the following functionality added:

- Importance sample filter option enabling the rendered image to be compatible with ML based denoisers like OIDN.

- Gaffer’s Output node parameters create both output layer and output driver attributes in 3Delight. This enables direct control of the rendered image bit-depth format, color space and filter (and a lot of other things). The old Quantize parameter is no longer used for this.

- Updated 3Delight output presets exposing the above functionality by default.

- Scene graph dl: attributes are exported as both geometry attributes and shader attributes in 3Delight. This enables 3Delight’s shaders to read any custom Gaffer attributes. Among other things, this is for ex. very useful when doing instancing where you would want to attach per-instance custom data.

- Camera overscan support.

- Automatic ASCII or binary NSI export from the DelightRender scene description output. This is based on the file name extension - .nsia for ASCII, anything else for binary output.

- Light groups (i.e. lightsets in 3Delight terminology) enabling rendering of one or multiple subsets of lights into separate images from a single render. This can be achieved by creating a custom Output string parameter called lightgroup and then adding a space separated list of light locations from the active scene.

While I tried to contain the main code changes to IECoreDelight/Renderer.cpp, in order to enable light groups I had to change the order of the scene data output processing in the GafferScene/Render.cpp file, so that outputs are processed last, instead of second after options (this is due to the lightset connection requiring the lights to be declared before the connection is made). I haven’t tested if this negatively affects Cycles or Arnold renders.

Finally, although this is not directly related to the changes in the Gaffer source code, I’ve noticed that two following tricks that can help a lot with creating complex shader networks in Gaffer. Since Gaffer doesn’t allow OSL surface shaders to be connected to OSL shader plugs, I simply changed the shader type directly in the compiled .oso file from surface to shader and this enabled complex layered surface shader networks to be built in Gaffer. Similarly changing a displacement shader from a shader type to a displacement type directly in the .oso file enables Gaffer to correctly recognize the shader as a displacement shader instead of a surface shader.

I’ve added an example of the adapted OSL shaders in the below Gaffer build, along with an AOV shader that uses single data inputs instead of arrays, which enables custom AOV creation and render with 3Delight.

Linux build (compiled under the suggested CentOS 7 environment – tested both with CentOS 7 and Ubuntu 22.04):

https://drive.google.com/file/d/1dQI5ximuA0wrgV6xsZ2KJTwUDkgxglCN/view?usp=drive_link

The adapted source code for the above build:

https://drive.google.com/file/d/1cX7EG2E23KzHnr6Zt0aYx4-yj_WijEN9/view?usp=drive_link

[John Haddon]

Thanks Goran, this sounds great! Would you be able to make some pull requests on GitHub so we can review you work and merge it to be included in the official releases? I'd suggest starting with a small pull request containing just one of your new features, as its much easier to review things in small focussed chunks.

Cheers...

John

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/001e0679-1a55-487c-8c40-4319be2864c8n%40googlegroups.com.

[Goran Kocov]

Sounds good! I have no previous Github experience unfortunately, but it won't be a problem to look into it and set things up so that I can do the pull requests. 

I have a few things I'd like to add to my custom build first - for ex. I just did an update enabling multilayer EXR writing which in turn enables full Cryptomatte AOV functionality straight out of 3Delight and I want to add basic lens shader support for the camera and procedural-like support for reading an outside NSI file or NSI lua script into the scene (similar to the old RIB ReadArchive call). Once that's done I'll update the public build and look into getting the changes into individual atomic pull requests. 

[Goran Kocov]

Here's an updated version of the 3Delight Gaffer integration providing the following updates besides the ones listed in the original post: 

- Support for direct rendering into multilayer EXRs along with fully working Output node presets for Crypomattes and also parameters for selecting the outputs that will be rendered into a single multilayer EXR. 

- Delight Option parameter for reading a NSI scene file or a Lua script through the NSI Evaluate function. This enables easy custom extensions of the generated NSI scene before it being passed to the renderer. 

- Update to the Gaffer function that reads the available OSL shaders which skipped certain very useful shaders like the area light shader and the decay and gobo light filters. The previously missing shaders will appear under the Others shader menu. 

I decided not to pursue adding support for lens shaders, partially since it required significant changes to how Gaffer deals with assigning shaders (lens shaders are connected directly to the camera object instead of through the geometryattributes) and partially because with the Evaluate function now available it's very easy to assign a lens shader using that mechanism. 

Updated Linux build:

https://drive.google.com/file/d/1BT68TPtv8dE9ykEswhy_gkSp93CEuXg0/view?usp=drive_link

Updated source code for the above build:

https://drive.google.com/file/d/1r2sTx8osiCp4bU_mkUZOFqlGS7cGilE0/view?usp=drive_link

With this being done as a next step I'll look into creating pull requests for the changes. 

Thanks! 

[Sachin Shrestha]

Hi Goran,

Thank you for the updated 3Delight integration with gaffer. It’s been a while since I used 3Delight so your build encouraged me to do some testing and in my basic gafferbot scene tests, I was quite shocked to see the difference between the render times of Arnold and 3Delight so I’ll be doing more exhaustive tests on production assets.

The multi-channel output is quite cleverly done and it works really well. I hope once your PR is merged, this feature also shows up on the GafferArnold side!

Thank you for sharing your work with the community.

Cheers,

Sachin

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/eb84865b-245e-49ab-a8bd-6105eb64759en%40googlegroups.com.

[Sachin Shrestha]

Hi Goran,

Wanted to report couple of issues that I noticed when testing the gaffer-3delight build on Linux (Rocky Linux 8.8).

1. If the prototype object that is being instanced is encapsulated with an Encapsulate node, the render cones out blank possibly because the encapsulated object is not being expanded when passed to NSI.
2. If I send the render to background when dispatching it from the DelightRender node, the batch render does not execute although the status in the jobs list shows it running. I noticed this error on the command line but not sure if it is related to this issue. If I execute this same render without sending it to the background, it works as expected.

WARNING : BackgroundTask : Unable to find ScriptNode for UVView.__textures
WARNING : BackgroundTask : Unable to find ScriptNode for UVView.__uvScene.out
WARNING : BackgroundTask : Unable to find ScriptNode for UVView.__textures
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/tmp/gaffer-3delight/gaffer-build/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/tmp/gaffer-3delight/gaffer-build/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/tmp/gaffer-3delight/gaffer-build/python/GafferDispatch/LocalDispatcher.py", line 230, in __backgroundDispatch
    self.__doBackgroundDispatch( self.__batch )
  File "/tmp/gaffer-3delight/gaffer-build/python/GafferDispatch/LocalDispatcher.py", line 238, in __doBackgroundDispatch
    if not self.__doBackgroundDispatch( upstreamBatch ) :
  File "/tmp/gaffer-3delight/gaffer-build/python/GafferDispatch/LocalDispatcher.py", line 291, in __doBackgroundDispatch
    process = subprocess.Popen( args, start_new_session=True )
  File "/tmp/gaffer-3delight/gaffer-build/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/tmp/gaffer-3delight/gaffer-build/lib/python3.7/subprocess.py", line 1482, in _execute_child
    restore_signals, start_new_session, preexec_fn)
OSError: [Errno 12] Cannot allocate memory

Thanks,

Sachin

[Goran Kocov]

Hi Sachin!

Thanks for the kind words and for testing out the updated 3Delight build. 

3Delight is definitely worth looking into when choosing a production renderer for an animation/VFX pipeline. My personal favorite feature is how short the time to first pixel is even on very large and complex scenes, which combined with Gaffer's speed at processing the scene makes for a very powerful combo. I was doing some tests on a scene with well over ten thousand non-instanced individual objects with a combined non-instanced polycount of several hundred million faces, rendered as subdivision surface geometry and the total time of Gaffer processing the scene to pass it to 3Delight and 3Delight initializing the scene to send it to the display driver was around a couple of minutes.

Regarding the two issues you mentioned, unfortunately encapsulated objects are not supported due to procedurals currently not being supported by 3Delight's Gaffer integration. Implementing both procedural support and supporting encapsulated objects should probably not be too complex, but for the time being my efforts will be focused on getting the current changes into the official repository.

I couldn't replicate the Python traceback with the dispatch and execution of the background render from the DelightRender node. I tested both in CentOS 7.9 (with and without a system level Python 3.7 installation) and Ubuntu 22.04 (without a system level Python 3.7 installation) and everything works as expected.  Sorry that I couldn't be of more help with this issue.

[Sachin Shrestha]

Hi Goran,

Thank you clarifying my doubts and questions. I will check the dispatch issue again as it could be a setup issue from my end.

Yes, I am certainly noticing a very good ttfp performance at my end too. Something else that I noticed within GafferDelight is that I am unable to plug the sky shader node to the 3delight environment light. The sky shader's output seems to be a closure/shader output plug but the env light expects a color input plug so I was wondering if this was by design and if Im am missing out some workflow details.

Thanks,

Sachin

[Goran Kocov]

Changing the OSL shader type from surface to shader should get the expected color output for the sky shader. You can do this even in the compiled .oso file by changing the following

surface dlSky %meta{string[1],tags,"environment"} 

into

shader  dlSky %meta{string[1],tags,"environment"} 

(I have attached the adapted sky shader for convenience). 

This is the same trick I mentioned in my original post and it can help a lot with being able to connect 3Delight OSL shader output plugs into the expected OSL input plugs (for ex. you can't do shader layering if you don't change the 3Delight surface shaders' type from surface to shader since Gaffer won't allow connecting a surface closure output to a color closure input, but it will allow a shader closure output for the same connection). 

[Goran Kocov]

Hi Sachin! Hi everyone! 

With most of the above updates already merged into the main Gaffer 1.4 codebase, here's an updated build that has the following additional new changes integrated (these are already merged into Gaffer's main branch as well): 

- John's fix for OSL surface shader outputs, which solved several issues like not being able to connect 3Delight's surface shaders into closure input plugs or the dlSky shader having an incorrect output plug. 

- Eric's added support for USD lights. 

- Fix for dlDisplacement being incorrectly applied as a surface instead of a displacement shader. 

- Reorganization of Gaffer's 3Delight shader menu adding some of the legacy 3Delight shaders, enabling custom shaders both with and without tags to be listed, adding a skyLight shader and a custom AOV shader. 

- Support for external procedurals as NSI scenes, Lua scripts and dynamic libraries / executables. 

I've also just implemented support for capsules as NSI procedurals (the encapsulated objects are exported as gzipped binary NSI files in /tmp and automatically read back into the scene). This is currently only supported on Linux and the code is a bit of a mess since I was trying different things, so it will take a bit time to cleanup for a proper PR, but you can try the functionality in the below Linux build (the Windows build is based on the GitHub artifact CI build, since unfortunately I don't have a local Windows build environment and due to this it's missing support for capsules)

This brings 3Delight's support in Gaffer to roughly 95% parity with Arnold with only a few features missing (like light linking and lens shaders that can be hacked around relatively simply in a few different ways). 

Linux: 
https://drive.google.com/file/d/1UE9fEIMF_RB_9WmljnVNxTb5RQuvr2kY/view?usp=drive_link

Windows:

https://drive.google.com/file/d/1cttP0Ou8BWAMV3Z1cp82cjsy7ZesgXUX/view?usp=drive_link

Hope you enjoy using the new (and old) 3Delight features in Gaffer! 

Goran

[Sachin Shrestha]

Hi Goran,

Thanks for the heads up and apologies for the delayed response! Will test out your build and report back any issues. The latest gaffer beta build seems to have some of your updates so hoping that soon you don’t have to maintain a separate build.

Cheers,

Sachin

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/37dcfb27-48e8-46a7-99d6-86070d820ccen%40googlegroups.com.