Import and assign shaders from an ass file

[Rupert Thorpe]

Hello,

If arnold shaders are baked out into an ass file, 

is there and way to import or assign them?

R 

[Brian Meanley]

Hi Rupert,

Just to clarify, do you mean by just importing the the ass file using the "ExternalProcedural" node, without actually manually re-building the shader-graph in Gaffer?

-Brian

MILK VISUAL EFFECTS

Threeways House,

40-44 Clipstone Street London, W1W 5DW

Tel: +44 (0)20 3697 8448

 

This message is intended solely for the addressee and may contain confidential and/or legally privileged information. Any use, disclosure or reproduction without the sender’s explicit consent is unauthorized and may be unlawful. If you have received this message in error, please notify Milk VFX immediately and permanently delete it. Any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of Milk VFX. 

By engaging in professional correspondence with Milk VFX, we may store your contact information for future reference. This is in line with Milk’s Privacy policy which can be found here. Milk Visual Effects is a registered limited company: 0844 1256. The registered company address is Threeways House, 40-44 Clipstone Street, London, W1W 5DW.

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/32cefc75-4e53-476c-905a-2ac33895181f%40googlegroups.com.

[Rupert Thorpe]

Hello,

Yes I am not expecting to get the shader graph in gaffer.

I just want to be able to use a shader that is in an ass file.

I did try using the ExternalProcedural node - but even if this makes the shader available to arnold, how can you do an assignment?

On Wednesday, 15 January 2020 18:38:34 UTC, Brian Meanley wrote:

Hi Rupert,

Just to clarify, do you mean by just importing the the ass file using the "ExternalProcedural" node, without actually manually re-building the shader-graph in Gaffer?

-Brian

On Wed, Jan 15, 2020 at 8:24 AM Rupert Thorpe <rth...@milk-vfx.com> wrote:

Hello,

If arnold shaders are baked out into an ass file, 

is there and way to import or assign them?

R 

MILK VISUAL EFFECTS

Threeways House,

40-44 Clipstone Street London, W1W 5DW

Tel: +44 (0)20 3697 8448

 

This message is intended solely for the addressee and may contain confidential and/or legally privileged information. Any use, disclosure or reproduction without the sender’s explicit consent is unauthorized and may be unlawful. If you have received this message in error, please notify Milk VFX immediately and permanently delete it. Any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of Milk VFX. 

By engaging in professional correspondence with Milk VFX, we may store your contact information for future reference. This is in line with Milk’s Privacy policy which can be found here. Milk Visual Effects is a registered limited company: 0844 1256. The registered company address is Threeways House, 40-44 Clipstone Street, London, W1W 5DW.

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffe...@googlegroups.com.

[Brian Meanley]

I don't believe there is a method in place currently to be able to assign shaders loaded from an ASS file to scene locations in Gaffer.  Someone correct me if I am wrong, but I believe that the geometry, and the shaders will need to be defined in the same ASS file for this to work inside of Gaffer.

-Brian

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/7c170bfd-7142-40a3-bee2-596777b682c8%40googlegroups.com.

[Don Boogert]

Perhaps I'm underestimating how much effort it would be but it shouldn't take long to write a simple .ass to gaffer converter in python, especially if you're not trying to solve the problem for all cases.

I could probably get you going if you post an representative shader .ass file. It script might be useful to others who want to convert maya authored lookdev to gaffer graphs.

Cheers

- Don

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAHvYvro1Sfv8k89ZZyJ2AjhAdo_GEssHVfuP%2BKbsTCTubts%2B3A%40mail.gmail.com.

[Rupert Thorpe]

Hi,

While this sounds very appealing - I think it may end up being unreliable in practice.   (I will post an example shader later when I have one that is representative) 

Also - a big motivation for me here would be the ability to use the same shader/lookdev across multiple applications.

This is possible between Houdini and Maya at the moment using shaders baked into an .ass file - and I would really want it to be the same.  

If you built a new shading network by parsing the file, how would you know for sure it was actually the same? 

R

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/7c170bfd-7142-40a3-bee2-596777b682c8%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAHvYvro1Sfv8k89ZZyJ2AjhAdo_GEssHVfuP%2BKbsTCTubts%2B3A%40mail.gmail.com.

[Tom Cowland]

Hey Rupert,

> If you built a new shading network by parsing the file, how would you know for sure it was actually the same?

Gaffer itself just assembles Arnold's own shaders into a graph. We don't do much in the way of conversion so what you see in the UI is pretty much what goes into the Arnold API. So as long as the shaders in your ass file are on the arnold shader path for Gaffer, and you recreate the shader names, connections and parameter values present in the ass file, I'd hope you'd be good. The only issues I can think off off the top of my head for cross app might be:

- MtoA for example may use additional shaders not included in the standard arnold distribution, so you'd need to make sure you put them on the shader path in your gaffer environment.

- IIRC, there are some data types from Maya ramps with connections that we don't have support for right now, or there may be other more esoteric parameter structures from other apps.

You could test conversion by writing an ass file back out from gaffer and comparing it to what you started with?

'best
Tom

[Brian Meanley]

Hi Rupert,

We've never used this workflow in our pipeline here, but I did just do a quick test and was able to get the shaders in the ass file (no geom) to be connected properly to a corresponding ass file with only geometry (no shaders).

Make sure the ExternalProcedrual nodes have a "namespace" attribute on it. I can't attach files from our work environment, so I am not able to upload an example or images unfortunately.

Also, make sure to follow this workflow when generating the ass files:

The parameter "Force Shader Assignments" (that is necessary for this workflow) appears in the export options UI and only appears when "Export Shaders" is disabled.

To export Stand-ins and shaders separately, the workflow would be :

Export your asset with only "Shapes" enabled with "Force Shader Assignment" enabled.
Export your asset again with only "Shaders".
Create two Stand-ins and load each of them.
Set the same Namespace string for both Stand-ins.

Let me know if that suits your needs.

-Brian 

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/1208BD33-6DD9-45C5-BDBA-EFE31EE0B1A6%40cinesite.com.

[Brian Meanley]

Sorry, that link is kinda botched. Here is the original link:  https://docs.arnoldrenderer.com/display/A5AFMUG/Stand-ins

[Rupert Thorpe]

Hi Brian,

Thanks for that info - I will investigate this today.  

Using a .ass file for the geo wont work for me since there wont be any representation of the asset in the gaffer viewport. -and also our assets will all be cached as alembic.

but maybe there is a way.

Hi Tom,

I see Maya nodes in the shader ass file eg:

MayaVolumeNoise

{

 name volumeNoise1

 placementMatrix 

 0.00100000005 0 0 0

 0 0.000250000012 0 0

 0 0 0.00100000005 0

 0 0 0 1

}

Arnold must understand this natively since this .ass shader works in houdini etc...  presumably there is similar nodes from all the applications.

can these nodes exist in gaffer?

I attach a simple shader .ass file.

R

To unsubscribe from this group and stop receiving emails from it, send an email to gaffe...@googlegroups.com.

[Brian Meanley]

Hi Rupert,

Sorry, I misunderstood your goal. I was assuming you were trying to link geo and shaders from two separate ass files, rather than the having the geo come from an alembic cache. Are you using ArnoldOperators in other DCC's to apply the look-dev to the alembic caches (judging by the "set_parameter" block in the attached ass file, I'm presuming this is the case)?  Gaffer does not currently support the ArnoldOperator set of nodes.

In regards to the following:

 

Arnold must understand this natively since this .ass shader works in houdini etc...  presumably there is similar nodes from all the applications.
can these nodes exist in gaffer?

Arnold core doesn't necessarily load these Maya-centric shaders (such as "MayaVolumeNoise") out of the box; however you are able to add all the Arnold plugins (MtoA, HtoA, SItoA, etc...) shader definitions to the Arnold search-paths, and Arnold will be able to translate these correctly across the various DCCs.  Gaffer works the same way. As long as those libraries are added to the search paths for Arnold, then you will have access to those shaders in your renders. You can verify that they are loading correctly in the UI by opening up a right-click context menu in the Graph Editor and go to "Arnold/Shader/Other".  If, for instance, the MtoA shaders are loaded correctly, you will see a bunch of shaders listed that are prefixed with "Maya".

-Brian

MILK VISUAL EFFECTS

Threeways House,

40-44 Clipstone Street London, W1W 5DW

Tel: +44 (0)20 3697 8448

 

This message is intended solely for the addressee and may contain confidential and/or legally privileged information. Any use, disclosure or reproduction without the sender’s explicit consent is unauthorized and may be unlawful. If you have received this message in error, please notify Milk VFX immediately and permanently delete it. Any views or opinions expressed in this message are solely those of the author and do not necessarily represent those of Milk VFX. 

By engaging in professional correspondence with Milk VFX, we may store your contact information for future reference. This is in line with Milk’s Privacy policy which can be found here. Milk Visual Effects is a registered limited company: 0844 1256. The registered company address is Threeways House, 40-44 Clipstone Street, London, W1W 5DW.

--

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/2eb115f2-b316-4488-9b18-22144c054bb6%40googlegroups.com.

[Don Boogert]

Here is a quick, dirty & largely untested script to read an .ass file and attempt to create shader nodes in gaffer:

https://gist.github.com/dboogert/d009fe2bfd13039aea9d8cde3fa61b35

Instructions:

Just download the file and run in the script editor, replacing the parameters to load_ass_shaders with the path to your .ass file & the name of the top level shader.

Warnings & caveats:

* It's very inelegant and only useful for educational purposes. It shouldn't take too long to see if this approach might be useful to you & to clean up the script.

* I'm not sure if I've created the connections correctly and in particular the color component connections

How useful would it be to *onboard* folk to gaffer if this was a standard feature?  Perhaps a python node could be created which references an .ass file and executes the script to construct it's graph structure on load or on demand?

Cheers

- Don

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAHvYvrp6_Ttx0WLRkv9ydRWcc1p4CXWy-RjkXzy0iZxcK01pEQ%40mail.gmail.com.

[Rupert Thorpe]

>Are you using ArnoldOperators in other DCC's to apply the look-dev to the alembic caches

Yes exactly.  

The ultimate goal would be to use the same lookdev in gaffer that we use in maya/Houdini.   

This would allow some shots to be completed in gaffer as a demonstration with very little overhead or risk.  and then sequences of shots where gaffers features are most helpful could be done with gaffer, plus turntables etc...   

I realise the operator nodes are not supported - I can rebuild the connections in gaffer so I can get around this, but would still need a mechanism to load the shader.

Rebuilding the network in gaffer sounds like it may actually be feasible - and I think many people would like this feature, but for me this would not be ideal - I want to be assigning exactly the same shader in gaffer that I do in maya/houdini - like if I delete/edit the .ass file then all applications should be affected the same way.

I guess I am asking for an "externalShader" node, that I point at an ass file and then can assign in gaffer just like it was a real shader - is this even possible?

R 

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/2eb115f2-b316-4488-9b18-22144c054bb6%40googlegroups.com.

[rupert thorpe]

Don Boogert!

You are my hero!

just used your script to rebuild a lookdev in gaffer - and it worked perfectly!!!!!!!!!11

doing this manually would not have been fun

[Aleks]

+1!

great info here thanks to all!

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/15adabfc-1ad1-4bc2-82f3-a2b07fa1ae64%40googlegroups.com.

[Heribert Raab]

+1
cool stuff !

Heribert Raab - Chat @ Spike

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/CAGZs%2BYHnnqa-PEKuaPhzC%2BNu_DRdCz10QvfG7Q6yF41t_g1Ykw%40mail.gmail.com.

[William Eguienta]

Hi,

Brian, I tried to reproduce the shading assignment from an .ass file but without success for now

can you share a file or copy paste the scene here please ?

Also, i'm trying to load an arnold usd scene as a standIn, without succes for now
In maya it's the exact same workflow that .ass, so i was thinking it will be the same in Gaffer

Anyone knows how to deals with usd standin with ExternalProcedural ?

thank you

[Viktor Holló]

Hey William,

For the usd question: you should change ExternalProcedural.name to "usd" so it creates an usd procedural instead of the default "procedural" which is for loading .ass files. 

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/4868b255-b7e9-4fcd-9154-6e1c395617f5n%40googlegroups.com.

--

	Viktor Holló Shot Lead | Pipeline TD PUPPETWORKS STUDIOS E-mail: viktor...@puppetworks.eu Web: www.puppetworks.eu

[William Eguienta]

Hi Viktor,

thank you, that was a quick & easy one :)

about assign shader from ass, my goal is the same as Rupert : having a reference shader in all DCCs

[Viktor Holló]

Brian's solution works if you are ok with having the geometries in .ass files.

If you are exactly in the same shoes as Rupert - so having geometry in .abc - then it seems the only current solution is to recreate the shader graph in Gaffer. That's what Don provided a "quick&dirty" script for: https://gist.github.com/dboogert/d009fe2bfd13039aea9d8cde3fa61b35
And as others reported, it works ;)

[William Eguienta]

about Don's script, I have gived it a try but it doesn't work for me when i try to import a 100% procedural (only with arnold nodes) from .ass made in houdini
I'm investigating on why,
Like Rupert, as I didn't need the graph I'm looking for a ass shader assignment to an alembic file

[William Eguienta]

here is an example of .ass from houdini that doesn't work with the script

- the name attribute doesn't match maya's one (easy to fix)

- some nodes inputs doesn't seems to exist, like for the ramp rgb node that only have a single input in gaffer (and maya) vs a variable numbers inputs in houdini

so it can't be translated as is in gaffer, an assShaderAssignment would do the job nicely in this case

You received this message because you are subscribed to a topic in the Google Groups "gaffer-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gaffer-dev/tqrK-b3VBjU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gaffer-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/a02c8986-e0ff-4eb6-b8c4-dcd78c93c8b7n%40googlegroups.com.

[Viktor Holló]

Well, we can just add "shader" custom attr. to locations. Load .ass file containing the shaders with ExternalProcedural and assign the shader to the geos with CustomAttributes "shader = shaderNameFromProc". In theory it should work.
But I tried, and it didn't :(

I checked that Gaffer uses "ai:" prefix for Arnold attrs. I managed to drive matte attr. by adding "ai:matte" custom attr (I know that there's an ArnoldAttributes node; I just experimented what can be done with CustomAttributes)
Then I saw that ShaderAssignment adds "ai:surface" attr. to the locations. I tried "ai:shader", "ai:surface", "shader" custom attributes, none of them worked. I guess Gaffer treats this as a special keyword and won't let me assign it manually as a simple string attr.

I'll try to look into the code to see what's happening exactly. Or John maybe you can hop in to enlighten us whether this workaround could work or not.

[Viktor Holló]

I must say I'm not a C++ expert, so I just hope I get it right: from the code it seems what I described is not working because Gaffer expects a ShaderNetwork to be connected to any of the following attributes of a location. ai:surface, surface, osl:surface, osl:shader.
And if it doesn't find a ShaderNetwork then it resets the shader attr.

So without custom Gaffer build there's no way you can assign a shader that is not present in the Gaffer scene.

What could possibly work is to create an OSL proxy/passthrough shader that just passes through the desired shader from the procedural. OSL would need to find our shader in the arnold scene by name during render time. My OSL knowledge is zero, so I don't know if it is even possible or how great the overhead would be.
Although in theory it could work, I don't feel that this is the right direction. Just brainstorming about the possibilities.

[William Eguienta]

Hi Viktor,

yes I've tried thoses methods too a couple of days ago
I guess that, as gaffer talks directly to the renderer, to make it works we need to mimic how arnold expect the shader to be assigned
witch is from the "set_parameter" part of an ass from what I see, like :

set_parameter
{
 name towerBridge_model_v001Shape_aiSetParameter
 selection "/towerBridge_GRP*" here is the object name
 assignment 3 1 STRING
  "shader='towerMat'" top shader name
  "subdiv_type='catclark'"
  "subdiv_iterations=1"
 declare is_target constant BOOL
 is_target on
}

if this set_parameter part can be done in a gaffer node to assign an ass shader to an object, then it should (maybe) works

[Viktor Holló]

set_parameter is an operator. And it has been stated that Arnold operators are not supported in Gaffer (on purpose)

--

You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/970b6e34-833a-4c6c-9732-1494e37dbd0fn%40googlegroups.com.

[William Eguienta]

yes, i'm aware of that but for many operators, we have a workaround in gaffer but not this one (and the materialX one too)

so I think we can have something that allow this kind of workflow

[John Haddon]

Sorry for my silence on this topic. I totally understand the desire to import existing lookdev into Gaffer, and to export lookdev from Gaffer to elsewhere. But rather than do this in an Arnold-specific fashion via `.ass` files, our plan is to add support for USDShade to enable interchange with a broader ecosystem. We haven't started work on this yet, but I hope to be able to share some progress on that front before too long. In the meantime, I think Don's importer is probably your best bet...

Cheers...

John

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/69dc6448-a577-4053-b2b8-5f8532e84e54n%40googlegroups.com.

[William Eguienta]

hey John,

Sounds good ! I'm not locked to .ass. In fact, the format itseft doesn't really matter (I'm evaluating the MaterialX options right now)
As long as I can publish a shader then load it in a 1:1 representation between DCCs,

so USDShade sounds perfect :)

thanks

[William Eguienta]

Hey John,
Coming back on this as I see that USDShade have been added to gaffer 0.61.2
Is there any example of how to load USDshaders in gaffer please ?
For now i don't see any attribute or something related ( in gaffer 0.61.11 )

Thanks

[John Haddon]

Hi William,

If you load a `.usd` file containing material assignments with a SceneReader, then those materials should appear in Gaffer as shader assignments on the relevant objects (which you could verify by looking in the Attributes section of the SceneInspector). In theory, if the assigned material is for Arnold, then it should just render directly without any intervention on your part. The majority of our testing so far has been in round-tripping materials in and out of Gaffer though, so there may be things we still need to work out for interoperability with other DCCs.

If you're able to share a small example file, I'd be happy to take a look...

Cheers...

John

To view this discussion on the web visit https://groups.google.com/d/msgid/gaffer-dev/a24b5732-8201-4bda-8c43-01b3805600een%40googlegroups.com.

[William Eguienta]

Hi John,

thanks, I haven't tried usd form gaffer for now, I have tested external DCCs only :
- USD from houdini  (test usd file here : https://fromsmash.com/8itkUVFEQZ-ct ) importing the same USD file in houdini shows the right shading

- the NVIDIA USD Attic Sample from here : https://developer.nvidia.com/usd (not my workflow, it's more for curiosity, to see if gaffer can read the shaders) - this scenes loads shaders too in houdini so i guess it's not specific to omniverse rendering

hope it's help

Cheers