"Deep" render pass renders at 100% but .exr not getting written

[Riccardo Cecchinato]

Hello dear Gaffer's devs,

I am having a technical issue with Arnold deep. I well know you are not responsible for specific Arnold-related issues, but with some luck you've hopefully already seen this one.

Long story short:

I am rendering a deep render pass - containing a .vdb - for allowing compositing to heavily modify a clouds layer.

In some of the shots where this vdb is present, certains Deep frames .exr won't get fully written to disk: the frame calculation arrives to 100% but its sub-job gets stuck in the farm apparently trying to unsuccessfully write the .exr file, until it times out (after 4 hrs).

Some additional info:

- The succesfully written deepexr are always around 58Mb of weight

- The "beauty" render layer (half precision .dwaa) containing the .vdb always renders without issues

- Only the deepexr render layer (full precision .zip or uncompressed) show the aforementioned issues, but only on certain shots, which however contains the same vdb as others that writes the deep .exrs without problems. 

- Trying to change parameters like bit precision, compression, augmenting depth tolerance, seems to not have an impact in solving this behavior.

- I am not posting any .log since no useful information are provided

- The issue seems to not be machine-related (the same machine can write one specific deepexr, and gets stuck on another frame)

- Few times requeuing a "failed" frame allowed it to get successfully written, for no apparent reason. 

Here is a screenshot of the Outputs node settings I am using for the deep render layer:

As usual, thank you in advance for your invaluable help!

Riccardo

[John Haddon]

Hi Riccardo,

This does sound familiar, but I'm having trouble recalling any specifics. I think it was an Arnold bug, but perhaps that has since been fixed. What Arnold version are you using?

Cheers...

John

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/ec12cf28-96a9-4205-8940-fa87e26c9d02n%40googlegroups.com.

[Riccardo Cecchinato]

Hello John! 

Thanks for the quick reply! We are on Arnold 7.4.2.1, the one before the very latest version.

Riccardo

To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVg%2B3ukzhX26H0vFY8A-JBM8kbivYd30-pLy6H4P-10C9Kg%40mail.gmail.com.

--

	Riccardo Cecchinato
	Lighting Supervisor
	stimstudio.com

[John Haddon]

I managed to dig up some info on the hangs we saw with deeps. They were caused by NaNs from the shading, and were fixed in Arnold 7.3.3.1. Not super helpful given that you're on a later version, but perhaps the NaN thing is something to follow up on?

Cheers...

John

To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/CAOA93KWRQrU-2N6KReXh%2BR-sfm%2B_7--HDM99%2BAasTMPuMer78Q%40mail.gmail.com.

[Riccardo Cecchinato]

John,

thanks for the lead! We'll try to dig into it with our TDs,

Have a great day!

Riccardo