Maya to Gaffer with USD

[Mathieu Maurel]

Hello everyone,

I've been testing Gaffer for 2 weeks now and first of all I want to congratulate you, it's a very powerful and pleasant software to use. The learning curve is very smooth, the documentation is very practical and the videos on your Vimeo helps a lot.

After spending some time recreating a scene from scratch in Gaffer, I am currently trying to figure out how to use assets from Maya - Arnold to Gaffer because before doing the entire LookDev in Gaffer, for now we are relying on USD to test in production and pass the assets from Maya to Gaffer. I'm mainly testing which nodes are correctly read by Gaffer, which ones are altered and which ones are simply ignored. 

So far almost everything works but my biggest concern is that aiRampRgb and aiRampFloat are not read by Gaffer, or to be more precise Gaffer only understands the first value and not the rest of the ramp. This is strange because aiRamp is I think in Arnold core (it is found in the documentation) but it is not in the list of Arnold shaders in Gaffer.

I saw that ramps are a known topic by searching in this Google Group but for now I see mostly answers talking about using OSL shaders, which does not solve our problem.

Have you some informations about that?

Some info to help you :
Maya 2022.3
Arnold MtoA 5.1.3
Arnold Core 7.1.2.0
Gaffer 1.2.0.0a1

Thank you
Mathieu

[John Haddon]

Hi Mathieu,

You're correct that `ramp_rgb` and `ramp_float` are core Arnold shaders. We hide them from the menus in Gaffer because we don't have a user interface for editing them. Instead we encourage people to use the OSL ColorSpline and FloatSpline shaders, which we have a user interface for, and which can be used in all the renderers Gaffer supports.

I suspect that building a native UI for `ramp_rgb` is a reasonably big task, but it should be much easier just to support loading and rendering from USD, where the UI isn't needed. Could you provide us with a very simple USD file containing a `ramp_float` and `ramp_rgb` and a reference render we can compare to?

Cheers...

John

41 rue de l'Echiquier - 75010 PARIS
http://www.brunchstudio.tv/

STATEMENT OF CONFIDENTIALITY
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. 
If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this e-mail and any attachment(s) is strictly prohibited. Please reply to the sender and destroy all copies of this message and any attachments from your system. 

Please consider the environment before printing this e-mail.

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/7c283769-5b83-42c4-9981-226946277ff0n%40googlegroups.com.

[Mathieu Maurel]

Hi John,

Thank you for your very quick answer!

Indeed being able to support aiRamp in USD rendering would be a huge plus for us.
Here is a USD file (it's a wetranfer link because I don't find how to attach a file) from Maya with 3 balls:
- the first one has a simple aiRampRgb
- the second one has a simple aiRampFloat
- the third one has an aiRampRgb with a green and red aiNoise in position 0 and an blue aiFlat in position 1 (just to check that the ramp is supported with textures or nodes plugged in)

I'm also attaching a Maya Arnold render of the expected result and just in case the network shading of the 3 balls and the outliner (juste the balls and a simple untouched aiSkydome) if ever needed to help.




Thanks,
Mathieu

[Mathieu Maurel]

Ok using my mail and not the Google Group I think I can join a file.

Mathieu

You received this message because you are subscribed to a topic in the Google Groups "gaffer-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gaffer-dev/9pcdwKzK4lE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVgL%2BpcGAdF0EJ5kYC2WtuAgp6%3D-ui3_UM0Ny4B3qss%3Ddbw%40mail.gmail.com.

[John Haddon]

Thanks - that's very illuminating. I don't suppose you could also provide an example where you make a connection into individual components of a color parameter? Like a float noise driving `color.r` on a `flat` shader? The problem is all to do with how connections are represented in USD, and I want to see what Maya is doing for that case too. My understanding was that USD didn't support component-level connections, or connections to individual array indices. Either I was wrong, or Maya has adopted its own syntax for that...

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CANJe6c9AQz339k6Gc%3DahhqrvGgjsjm0o%2BCm%3DTUhUz%2BSN0KqtWw%40mail.gmail.com.

[John Haddon]

I've attached a Gaffer file where I've used a Python expression to fix up those connections into a form that Gaffer/Arnold can understand. Could you check that the output looks as you expect? This is just a hack, and would have terrible performance for a large scene due to using Python, but hopefully is of some short-term help. The next step is to figure out if these connections are standard or non-standard, and then implement some appropriate conversions in C++ somewhere...

Cheers...

John

[Mathieu Maurel]

Thanks - that's very illuminating. I don't suppose you could also provide an example where you make a connection into individual components of a color parameter? Like a float noise driving `color.r` on a `flat` shader? The problem is all to do with how connections are represented in USD, and I want to see what Maya is doing for that case too. My understanding was that USD didn't support component-level connections, or connections to individual array indices. Either I was wrong, or Maya has adopted its own syntax for that...

Yes I can provide a lot of things to help you!
I Join a new file USD_aiRamp_2.usd. It's a new ball (plus the old ones) with an aiFlat driven by 3 aiNoise (different scale in R G and B). I hope it's what you're asking for?

I've attached a Gaffer file where I've used a Python expression to fix up those connections into a form that Gaffer/Arnold can understand. Could you check that the output looks as you expect? This is just a hack, and would have terrible performance for a large scene due to using Python, but hopefully is of some short-term help. The next step is to figure out if these connections are standard or non-standard, and then implement some appropriate conversions in C++ somewhere...

Unfortunately I have only the first value of the aiRamp, as on my previous tests. I see that in the sceneRender of your scene there is a .usda but I put my .usd previously provided, I hope it's not because of that that it doesn't work?

I'm attaching my Gaffer scene in case I did something wrong (I'm a 2 week old newborn on Gaffer so it's clearly possible that the mistake is on my side)




Thanks,
Mathieu

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVg%2B9vZe_ed_zi0_TURNMeZtR4mJR3whUk1MiwvUfF9_Njw%40mail.gmail.com.

[Mathieu Maurel]

I forgot to mention that I use the Arnold USD exporter from Maya and not the default Maya USD (because Maya USD does not export shaders).

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVg%2B9vZe_ed_zi0_TURNMeZtR4mJR3whUk1MiwvUfF9_Njw%40mail.gmail.com.

[eric...@gmail.com]

What version of Gaffer are you using for these latest tests? John and I were looking at it this morning and we think you may have success with the latest alpha version released on Friday  Release Gaffer 1.2.0.0a2 · GafferHQ/gaffer (github.com)

That includes a fix that addresses the `WARNING [setParameter] Unsupported value type "InternedStringData" for parameter "type" (expected StringData).` Testing with that latest release we get the results you posted above, with the gradient and additional noise mixed in.

Can you give that version a try and let us know if it fixes it for you?

[John Haddon]

On Mon, Jan 16, 2023 at 3:28 PM eric...@gmail.com <eric...@gmail.com> wrote:

That includes a fix that addresses the `WARNING [setParameter] Unsupported value type "InternedStringData" for parameter "type" (expected StringData).` Testing with that latest release we get the results you posted above, with the gradient and additional noise mixed in.

To be extra-double-triple-clear, we see the ramps working when we use 1.2.0.0a2 and the little workaround in the `.gfr` file I attached earlier. If you can confirm that works for you too, then I can make sure this all works out-of-the-box in the future.

[Mathieu Maurel]

Hi Eric,

I just tested the scene with the 1.2.0.0a2 and here are my results:
- Without your Box and the Python script the aiRamp are well read by Gaffer but not the one with connections in the aiRamp, only the ones with simple values.
- With your Box and the Python script everything works perfectly !

  

I also tested the last USD sent to John with the 4th ball (which has an aiFlat and 3 aiNoise connected in the R G and B) but unfortunately it doesn't rendering.



I hope all this feedback will be useful to you, don't hesitate to ask me anything to help you.

Thanks for your reactivity,
Mathieu

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/09cd6ca0-a0ee-4b50-8c49-3cf250aeddabn%40googlegroups.com.

[Mathieu Maurel]

Hi John and Eric, it's me again.

I'm just here to say a HUGE thank you for the quick change between 1.2.0.0a1 and 1.2.0.0a2 because it fixed a lot of things that helps us a lot.

I have been comparing the Maya and Gaffer transition with Arnold USD for several days to see which nodes are supported, which ones give a different result and which ones don't.
And the gap between version a1 and a2 fixed 99% of Arnold nodes!

I put you the comparison renders as well as the test usd and the gaffer scene just in case.

 



And also a list of tested nodes (this is the result with Gaffer 1.2.0.0a2) :

• Line 1. Maya Nodes and aiRamp
  1. ✅ aiFlat
  2. ✅ Maya Ramp
  3. ✅ Maya Ramp with aiFlat
  4. ✅ aiRampRgb
  5. ⚠️ aiRampRgb with aiFlat - Not the same result
  6. ❌ Maya Bulge
  7. ✅ Maya Checker
  8. ❌ Maya Cloth
  9. ❌ Maya Fractal
  10. ❌ Maya Grid
  11. ❌ Maya Noise
• Line 2. Arnold Texture
  1. ✅ aiFlat
  2. ✅ aiCellNoise
  3. ✅ aiCellNoise with aiFlat
  4. ✅ aiCheckBoard
  5. ✅ aiCheckBoard with aiFlat
  6. ✅ aiCurvature
  7. ✅ aiFlakes
  8. ✅ aiNoise
  9. ✅ aiNoise with aiFlat
  10. ✅ aiThinFilm
  11. ✅ aiTriplanar
• Line 3. Arnold Shader
  1. ✅ aiAmbiantOcclusion
  2. ✅ aiCarPaint
  3. ✅ aiLambert
  4. ✅ aiLayerShader
  5. ✅ aiMatte
  6. ✅ aiMixShader
  7. ✅ aiShadowMatte
  8. ✅ aiSwitch
  9. ✅ aiUtility
  10. ✅ aiWireframe
  11. ✅ aiRaySwitch
• Line 4. Arnold Utility Math
  1. ✅ aiAbs
  2. ✅ aiAdd
  3. ✅ aiAtan
  4. ✅ aiCompare
  5. ✅ aiCross
  6. ✅ aiMax
  7. ✅ aiMin
  8. ✅ aiMultiply
  9. ✅ aiDivide
  10. ✅ aiRandom
  11. ✅ aiTrigo
• Line 5. Arnold Utility Shader
  1. ✅ aiBlackBody
  2. ✅ aiBump2d
  3. ✅ aiClamp
  4. ✅ aiColorConvert
  5. ✅ aiColorCorrect
  6. ✅ aiLayerFloat
  7. ✅ aiLayerRgba
  8. ✅ aiNormalMap
  9. ✅ aiRange
  10. ✅ aiRoundEdge
  11. ✅ aiFacingRation
• Line 6. Arnold Utility Shader (suite) and Misc
  1. ✅ aiShuffle
  2. ✅ aiSpaceTransform
  3. ✅ aiUvProjection
  4. ✅ aiUvTransform
  5. ✅ aiFloatToRgba
  6. ✅ aiFloatToInt
  7. ✅ aiColorToFloat
  8. ✅ aiRgbaToFloat
  9. ✅ aiTwoSided
  10. ✅ aiClipGeo
  11. ✅ aiPhysicalSky
• Line 7. Arnold Projection
  1. ✅ aiCameraProjection

So once again thank you very much for your work.
Mathieu

--
You received this message because you are subscribed to a topic in the Google Groups "gaffer-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gaffer-dev/9pcdwKzK4lE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gaffer-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVgLqRiGFGZ7oZddCO7nnRCGPrXR5ox4%3Dps27%3DHTaP6fXxw%40mail.gmail.com.

[John Haddon]

Thanks for the feedback Mathieu, that's great news!

I've fixed the ramp-array-index connections in our USD loader, so you shouldn't need the Python-expression workaround in the next version of Gaffer : https://github.com/ImageEngine/cortex/pull/1324.

I've also looked into the color-component connections (the fourth sphere), and determined that the USD connection that Arnold-USD was writing in that case was invalid. This was already reported as an Arnold-USD bug, and fixed as well. I don't know how Arnold-USD is packaged for Maya, but the fix is mentioned in the release notes for 7.1.3.0, so maybe an Arnold upgrade could sort that out?

Cheers...

John

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CANJe6c8B2_kuAXJafZUOd8AN0kD8b-t6Fb%2BHwLcKykZbdkxFYg%40mail.gmail.com.

[Mathieu Maurel]

Hi John,

Great news!
Thanks for all this updates.

Take care,
Mathieu

[aurora]

Hi John, 

Using this expression method, is it possible to create or modify shaders inside the shader network? For example creating a brand new rgb_ramp shader modifying the ramp colors and connecting it to the base color of the output shader. I am specifically looking at doing it with cycles Shader, but I would think the method is the same for others.
Thanks

A

[John Haddon]

Yes, that is possible - you can basically do anything you want in the Python expression, because it has full access to the underlying ShaderNetwork API that Gaffer uses. But in this case, it's much easier to achieve using the ShaderTweaks node, which can insert new shaders as well as modify existing parameters. I've attached an example.

Are you interested in this because ramps aren't importing from Blender/USD correctly? If so, we'd like to fix that, so example data would be most welcome...

Cheers...

John

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/2cb43e4b-3deb-44e0-bfb1-23bba7613b98n%40googlegroups.com.

[Aurora Films]

Thank you John.

Yes my questions is real are to some issues with Blender exports.

I’ll create a separate thread for this.

Cheers!

A

Sent from my iPhone

On Jan 26, 2023, at 12:40 AM, John Haddon <jo...@gafferhq.org> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAB8pVg%2BpSdcxF0AyO7cafKyX3V0xj26WW_1ZSqyDh_YxSvdo7Q%40mail.gmail.com.

[Alex Fuller]

Hi Aurora,

From memory the ramps were tricky to support at Tangent Animation too going from Blender->Houdini/Hydra as it is just 256 float array values at that stage with the curvature+interpolation "baked" from Blender, so making a way to tweak them with 256 knots wasn't the most fun thing in the world. A pass-through without the need of tweaking after probably would be fine though.

https://developer.blender.org/diffusion/B/browse/universal-scene-description/source/blender/io/usd/intern/usd_writer_material.cc$1666

A solution that I was proposing at the time at least internally was a portable (non-GPL) spline library to live in Blender and the Hydra delegate. I'm not sure if USD has one now, but the one out of Cortex would be a great fit for Gaffer compatibility, something to discuss with the Blender people probably...

[John Haddon]

On Tuesday, January 17, 2023 at 10:40:27 AM UTC John Haddon wrote:

I've fixed the ramp-array-index connections in our USD loader, so you shouldn't need the Python-expression workaround in the next version of Gaffer : https://github.com/ImageEngine/cortex/pull/1324.

This fix is now available in Gaffer 1.1.8.0 and 1.2.0.0a3 : https://github.com/GafferHQ/gaffer/releases/tag/1.2.0.0a3.

[aurora]

Thank you John.