Import Shading from Maya and Cycles, Shot management in Gaffer

[Sala JP]

Hello,  I was introduced to Gaffer just last week, and I was amazed by it and the community behind it.

But at the same time, I found out that if I lack the knowledge of scripting its very hard for me to adapt the software that easily.

For now, this is what I would like to do with the software and this is the difficulties I found.

The basic idea is to export scenes with shaded objects using Maya / Arnold or Blender / Cycles, and export them to Shotgrid with basic slapcomp.

Not sure yet if I want to export the lights from Maya or Blender to Gaffer or Re-Light the scenes in Gaffer.

 

Since all the look div is done in Maya and Blender, it would be a lot of work to re-construct the whole thing in Gaffer.

 

So I did some research on how to export objects with Shaders from Maya or Blender, and then import them to Gaffer, but I didn’t find a clue anywhere.

It looks like I need some sort of scripting or coding, which I have zero knowledge of that.

So I tried GPT Chat, and I asked it for coding, which none of it worked, maybe I was applying it wrong or the codes weren’t that good or something.

Then luckily I stumbled on this Gaffer Group hopping to learn the software with your help and support

 

For now, this is what I hope to set it up.

1-       How to import shaders from other software such as Maya (Arnold) and Blender (Cycles) automatically to Gaffer with the correct parameters and inputs.

2-       Is it also possible to import Cameras and Lights with the correct settings from other software?

3-       I have seen an amazing video on Vimeo on how to use Gaffer and in that explainer video they showed it is possible for Gaffer  to have shots embedded in Sequences and so on, that would be wonderful if I could have that.
of course, I browsed the internet, YouTube, and Vimeo to find out how its made, but without success.

4-       It looks like I have trouble getting Gaffer to use my GPU. I’m not sure why, but it’s using my CPU for now.

5-        

Thank you in advance!

[John Haddon]

On Mon, Nov 11, 2024 at 10:00 AM Sala JP <doubl...@gmail.com> wrote:

1-       How to import shaders from other software such as Maya (Arnold) and Blender (Cycles) automatically to Gaffer with the correct parameters and inputs.

I would recommend exporting using USD, and then importing to Gaffer using a standard SceneReader node. There may be some rough edges, particularly with Blender/Cycles, but that's for sure the best starting point.

2-       Is it also possible to import Cameras and Lights with the correct settings from other software?

Cameras should be fine via USD also. If you're using UsdLux lights then those will also import and render in Gaffer just fine. I haven't tested Arnold/Cycles lights this way though, and suspect they won't work, but let us know what you find.

4-       It looks like I have trouble getting Gaffer to use my GPU. I’m not sure why, but it’s using my CPU for now.

Use your GPU for what? Rendering with Cycles? Arnold? Something else? We need more details about what you're trying to do, what you've tried, and what happened...

 

Cheers...

John

[aurora]

I have looked into converting Blender Cycles Shading Networks to Gaffer using USD. Definitely possible.  
Blender USD branch will give you option to store shading networks into USD files. The resulting USD Networks are not fully readable by Gaffer, so you need some scripting to help with the translation.
There are some limitations due to Blender 's implementation of Cycles, but I would say you can get 90% there with scripting an then manual tweaking, depending on the complexity of the network.
You can look in this thread for some tests: 
https://discord.com/channels/722635732384219146/1212119740043108413/1240673905291825172
That is actually a bit outdated since last time I worked on it. Eventually I'll put the test in GitHub.

Since blender USD branch is not always current with the latest version, I think a better approach is to take advantage of the USDHook method introduced in Blender 4. It allow you to script you own conversion or material networks as the USD file gets exported, and probably do it in a Gaffer friendly fashion. This way you dont rely on the sporadic USD branch. I started some work on that but got nothing usable yet.

Another method that have been on my radar is to take advantage of the new MaterialX export options in the latest version on Blender. It converts the cycles shaders to MaterialX and stores them in the exported USD. I am not sure how MaterialX is handled in Gaffer, or even if it possible to use Cycles to render them. 

[Sala]

Hello John and Aurora,

Thank you for the response

John, I tried to import an 3D object via USD from Blender to Gaffer, the object seems to be transferred correctly but the camera seems lost some of its settings such is Focal Length.

 I had a problem with the GPU rendering before, but now it seems to be working fine, just crashing sometimes.

[Sala]

Hellow Aurora,

Thank you for the link, I followed the link to Discord, then I tried to download the Collect_blender_materials_example.zip but the link is dead saying “files doesn't exit”. I attached a screenshot.

About USDHook, unfortunately I don’t know how to code or even where and how it functions, since I don’t know coding im worried it will take me more time and make more mistakes.

For now, I did a test importing a 3D object that is shaded in Blender and exported via USD, screenshot attached.

When I imported to Gaffer this is what I noticed.

1-the object seems to be imported correctly

2-the textures embedded in the USD is displayed wrong (Tank Screenshot is attached)

3-camera embedded in the USD seems to be in the correct place, but the settings seems to be wrong, for example the Focal Length.

4-in another test (file name is Long_house), the texture embedded in the USD didn’t preview the Alpha on the roof (screenshot attached)

5-when I rendered I received these errors (Gaffer files are attached for the Tank and Long house tests) (the errors disapplied below)

My Questions are Questions:
1-can I tweak the texture embedded in the USD? Lets say I would like to reduce the Specular or Roughness for example.

2-can I tweak the cameras embedded in the USD? Like fixing the focal length for example? Or tweak the animation.

3-for those texture errors, that I mean I need to rebuild then in Gaffer?

WARNING [IECoreCycles::ShaderNetworkAlgo] Couldn't find socket output "color.r" on shaderNode "shader:373885d2a0a6236d387920ddb792593d:Image_Texture_003"

WARNING [IECoreCycles::ShaderNetworkAlgo] Couldn't find socket output "color.r" on shaderNode "shader:0589479bc84f07b023c78c306926dbce:Image_Texture_002"

WARNING [IECoreCycles::ShaderNetworkAlgo] Couldn't find socket output "color.r" on shaderNode "shader:373885d2a0a6236d387920ddb792593d:Image_Texture_001"

4-what should I do for this error?

WARNING [IECoreCyles::GeometryAlgo::convertPrimitiveVariable] Primitive variable "sharp_face" has unsupported type "BoolVectorData".

In conclusion,

1-if I used the script you provided in Discord, would that solve the issues?

2-I assume big studios don’t rebuild their shaders in Gaffer, whats their solution to that?

Note
the Tank files is too heavy for this post, its about 24GB, so i'll upload it here for you to download it here just in case.

https://we.tl/t-z7iUR8GjjL

Thank you in advance

[aurora]

Seem like you are using the official stable Blender version. That will only give you the option to export Basic USDPreview Surface network - and MaterialX - but not Cycles networks. 

The USD Brach Blender build will allow you to export the Cycles Network to USD. 

Here are some files you can test:

https://drive.google.com/file/d/15swkGiKyZX6795HMeoxHhB7JqElizrBp

If you export you scene with the USD Branch you may be abler to run the usd file thought the CollectMaterials box and get it to rebuild your shaders inside.

You can also post your Tank scene .blend file an I can test it for you.

A

[Sala JP]

hello again aurora, but when i came back to the message i couldnt find it for some reason, so i'll reply you from my memory and sorry if i didnt answer one or two of your quesions!

I did a test by importing a gras with basic shader from Blender to Gaffer with the script you sent me, here how it went

1-I copied and pasted the script to a new Gaffer script, imported the Grass.usdc successfully.

2-after adding the grass.usdc i clicked on the button Create Cycles Materials, it gave me an error

3-I dug deep into the script to find that MATERIAL_NETWORKS Box with the material CollectMaterials Node is still pointing to the old object (Camera.USDC), same thing with the MATERIAL_SETS Box as shown in the screenshots below.

Can you please show me how to fix this issue?

Also, it seems in your last message that i cant find, you mentioned something about Exporting the Cycles Shader Network from blender needs an add on to download or there's a work around to achieve that? 
i checked with Blender website and the internet i couldnt find a solution for that, 

can you point me to a tutorial where i can export the Shader Network from Blender to be imported into Gaffer with your script?

Below you'll find the Scenes for the Tank and the Grass, sorry i missed it last time.

Click here to download 

appreciated, thanks!

[aurora]

You really need the USD branch fro this to work. If its not available for download anymore you may need to compile it.
The error  on your second screenshot is caused by the script not finding the material location prim path (defined in the tool) in the USD stage. My code doesn't nicely handle the error, hence why you get such a cryptic message.
Additionally, seem like the texture issues you were having are caused by textures having the same filename for different materials. If you export the textures from Blender they dont respect the original path and all get saved in the same folder, overwriting each other. A workaround could be to disable the texture export option in blender and let the shader point to the original path:
May play around with you scene a bit more to export a better example for Gaffer.
A

[Sala]

thanks Again, 

i checked the script again, didnt know where i can point the materials to the correct folder, it seems that the script i have is fixed to the Camera Model Textures.

is there's a way to point the script to the correct texture prim folder and press the Create Cycles Materials again ? 

also, i didnt get this comment "You really need the USD branch fro this to work. If its not available for download anymore you may need to compile it."

you mentioned before that the stable version of blender cant export the Shader Network with USD, is this what you ment? 

thanks

[Aurora Films]

When exporting from Blender you can define the prim path where the material get stored. 



Yes, thats what I mean. Stable version of Blender cannot export cycles networks.
Even with the USD Branch, not everything is 1:1. An example of this is that in your .blend scene the Mix shader operation is set to "Multiply", for whatever reason the parameter doesn't get stored in the USD material. 

You could fix this in Gaffer onee you rebuild the networks, or change the share to do the same differently. I have included a test of the latter in the link below. 
https://drive.google.com/file/d/15bh9HzUpgPdoPqxp5h0Fpv6xYqMSkDpW/view?usp=sharing

This is what I mean that this method will get you only 90% there, with tweaks need to be done in Gaffer for final results.

A

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/119d2e15-e382-4e9f-855b-470bd6ce95can%40googlegroups.com.

[ds]

thank you again Aurora, 

the Gaffer file Samples you sent me works fine, clearly Gaffer is a great tool, its way too smart for me to start a production with it at the moment i have to spend more time to study it. 

i wish if there's tools out of the box to import textures and deal with all the issues mentioned, your tool is great and im sure its a big time saver, i'll have a deeper look into it and i'll keep an eye on Gaffer for the up coming releases. 

Thanks again and best of luck! 

To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/CAHdutf3o-9VnWo54MNocw3PDY%2BDYvxkVoYDCXbeJyaBdPfh3Cw%40mail.gmail.com.

[Aurora Films]

Good luck to you!

A

To view this discussion visit https://groups.google.com/d/msgid/gaffer-dev/CAMa%2BB6FEoAT2Yusd4Cx%2BGJQLk4prAuTYW6rQbqnfWHO92SfvHQ%40mail.gmail.com.

[Sala JP]

much appreciate it :)