"tags" in gaffer

[Sachin Shrestha]

Hey folks,

When we use a SceneReader node, it provides an attribute called tags which is supposed to load only the elements of the scene specified in the tag. How do we export these tags into alembic from maya or other dccs? Is it a specially named extra attribute on the shape or the transform? We were internally discussing in our gaffer test group on ways to automate certain things in gaffer pipeline and this seemed like a nice way of filtering scenes. So, in pipeline we could tag assets before exporting them as "characters", or "renderable", or "ignore", etc.

Also, in general, is there a concept of creating tags within gaffer or are we always supposed to use sets for that? In our current lighting/lookdev package, we can add tags to any element in the scene with the convention "@char" for something like characters. We can then refer to this tag downstream and filter all elements or apply overrides, etc. based on this tag. We are currently using sets only to define the shader bindings within each asset but it may cover this use case well as well? The only issue being that we need to always set the filter based on paths which could be subject to change. If we could read in our extra attributes and use those to auto-magically drive sets or tags, then we could build our pipeline around that. I didn't find any previous example on the mailing list for this.

Cheers,

Sachin

[Andrew Kaufman]

Hey Sachin,

Tags are part of the Cortex SceneInterface API. We currently only export Tags (from Maya, Houdini, etc) when writing SCC files, but we are actually planning to add ABC support soon by converting Tags to AbcCollections on export. That would require using IECoreMaya or IECoreHoudini to create your ABC files though.

You should also be aware, we are in the process of replacing Tags with Sets (PathMatchers) in the SceneInterface, so its probably best for the mid/long term if you just think about AbcCollections in your file and Sets in Gaffer. I'd imagine the Tags plug on the SceneReader will disappear eventually. Once AbcCollections are loading as Sets in Gaffer, I think the workflows you're envisioning will be quite natural.

Hope that helps clarify things.

Andrew

--
You received this message because you are subscribed to the Google Groups "gaffer-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sachin Shrestha]

Hi Andrew,

Thanks for the clarification. On a similar note, if we were to add certain extra attributes on objects in Maya for our pipeline purposes and export them into the alembic cache, is there a way to use those extra attributes in gaffer after loading the geometry?

Thanks,

Sachin

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+...@googlegroups.com.

[Andrew Kaufman]

If you’re referring to location based attributes (as opposed to primitive variables on the mesh surface), Don opened a PR for alembic support recently:

https://github.com/ImageEngine/cortex/pull/816

Once that makes it into a Gaffer release you should be able to use your Abc attributes just like any you could create with a StandardAttributes or CustomAttributes node. Note that the attributes will likely have a “user:” prefix once loaded in Gaffer.

If you were referring to per-vertex or per-face values on the mesh, we already load those primitive variables from Alembics, and they appear on the object (rather than the location). You can read them in shaders, and some SceneNodes use them for processing as well (see Seeds or Instancer).

Don also opened a PR for a new UI pane to view the primitive variables so you can validate the data more easily:

https://github.com/GafferHQ/gaffer/pull/2699

Cheers,

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to gaffer-dev+unsubscribe@googlegroups.com.

[Sachin Shrestha]

Got it. Thanks Andrew!

Cheers,

Sachin

[Sachin Shrestha]

Hi Andrew,

It seems that the PR you linked above for alembic attributes made it into the recent gaffer releases. I tested it by exporting custom attributes on shape nodes from Maya into alembic and they seem to show up for me in gaffer in the Scene Inspector. This only works for attributes exported from shape nodes and not the ones on transforms so is that by design?

Also, I exported a vector attribute called myColor which I tried using in an arnold shader network via oslIn node where I supplied the attribute name as user:myColor but I get the following error when trying to render the scene:

WARNING : setParameter : Unsupported data type "V3dVectorData" for name "myColor"

I get the same erroe when I try to use a float attribute:

WARNING : setParameter : Unsupported data type "DoubleVectorData" for name "myDiffuse"

Looks like the data type conversion is not handled currently? And it doesn't seem to matter if I prefix the attribute name with user: as it reports the same error without the prefix. I have attached a very simple repro that illustrates this issue. This was tested with 0.48.1.0.

Also, based on my original query in this thread, how would I use custom string attributes to drive something like sets in gaffer? F.e. if I export custom string attribute called "setName" on all objects and then define the gaffer material set name in these attributes, how would  be able to use them to drive the set membership in gaffer?

Cheers,

Sachin

[Andrew Kaufman]

> This only works for attributes exported from shape nodes and not the ones on transforms so is that by design?

I would have expected it to be the opposite to be honest. In Cortex/Gaffer we don't distinguish between a transform location and a shape location. All locations have a transform and can optionally have a shape (object). Looking at your example abc, it seems you've exported what we call PrimitiveVariables rather than Attributes. In Cortex/Gaffer the distinction is important. An attribute is a property of a location, while PrimitiveVariables are related to the topology of an object. Some nodes in Gaffer only operate on PrimitiveVariables and some only operate on Attributes. Since all of your custom PrimitiveVariables in the example are Constant (one value for the entire shape as opposed to one value per vertex/face/face-vertex), it would be preferable if you could export those as Xform attributes instead.

In the translation layer, we interpret Xform User Properties as Cortex/Gaffer Attributes, and we interpret shape ArbGeomParams as PrimitiveVariables. I'm not too familiar with debugging Alembics, but using abcecho on your example file, it seems we are interpreting the data correctly, in that the myColor/myDiffuse/mtlName properties are listed as arbGeomParams within the PolyMesh schema, while all of the Xform schemas are listing empty userProperties.

> I exported a vector attribute called myColor which I tried using in an arnold shader network via oslIn... I get the same error when I try to use a float attribute

I think there are a couple problems. First is that both PrimitiveVariables are doubles (V3dVectorData and DoubleVectorData), whereas I think our OSL ShadingEngine only supports floats (V3fVectorData or FloatVectorData). There is a special case to convert double->float in the code, but if I'm reading it correctly, it doesn't work for VectorData currently. So if you could export float data (IV3fGeomParam) instead, that might just work.

It is a bit odd that these variables are VectorData in the first place, since they are each a single Constant value. Was that intentional to make then arrays/vectors? Or could that be a bug in our translator? I did noticed we have a test that is explicitly guaranteeing a Constant IntVectorData PrimitiveVarible, so I'd have to check on the history a bit to understand why we're doing that. Maybe there just isn't a good way to export non-vector data from Maya?

> Also, based on my original query in this thread, how would I use custom string attributes to drive something like sets in gaffer? F.e. if I export custom string attribute called "setName" on all objects and then define the gaffer material set name in these attributes, how would  be able to use them to drive the set membership in gaffer?

I think you're going to need to export AbcCollections in order to get Sets in Gaffer. I don't know if/how to export AbcCollections from Maya though. One would hope Maya Sets just get exported as Collections by default, but maybe that is not the case?

Note that the Abc Collections -> Gaffer Sets translation still hasn't been implemented, but when it is, the collections will just come through as Sets automatically. We had to de-prioritize that unfortunately since we're not using Alembic in production much and other things have taken precedence...

Hopefully all that info is useful...

Cheers,

Andrew

--

[Sachin Shrestha]

Hi Andrew,

Thanks for the detailed response. 

 

In the translation layer, we interpret Xform User Properties as Cortex/Gaffer Attributes, and we interpret shape ArbGeomParams as PrimitiveVariables. I'm not too familiar with debugging Alembics, but using abcecho on your example file, it seems we are interpreting the data correctly, in that the myColor/myDiffuse/mtlName properties are listed as arbGeomParams within the PolyMesh schema, while all of the Xform schemas are listing empty arbGeomParams.

I see that the Maya alembic exporter is doing something different than what is expected (or may be its default behavior is designed this way). I was actually exporting the attributes on the xforms first but since they did not show up in gaffer, I tried them on the shape nodes instead. As you rightly pointed out, these get interpreted as prim vars and not attributes. Debugging the exported alembic a bit further, if you were to import it back in maya, you'll notice that there is an attribute "mtlName" on the top xform node "model_grp". However, for whatever reason the Maya abc exporter exports even xform attributes as arbGeomParams and not arbGeomParams. And since gaffer reads arbGeomParams on shapes, the xform ones are ignored. Here's a snippet from abcecho:

Object name=/model_grp
  CompoundProperty name=.xform;schema=AbcGeom_Xform_v3
    CompoundProperty name=.arbGeomParams;schema=
      ArrayProperty name=mtlName;interpretation=;datatype=string;arraysize=1;numsamps=1
    CompoundProperty name=.userProperties;schema=

The mtlName should be a user property and not a geom param. I suspect I'll need to export the abc via alembic python api so that I can structure it the way gaffer expects.

 

It is a bit odd that these variables are VectorData in the first place, since they are each a single Constant value. Was that intentional to make then arrays/vectors? Or could that be a bug in our translator? I did noticed we have a test that is explicitly guaranteeing a Constant IntVectorData PrimitiveVarible, so I'd have to check on the history a bit to understand why we're doing that. Maybe there just isn't a good way to export non-vector data from Maya?

I'm not doing anything special with those attributes. I just added them via Maya's Add Attribute UI on shape nodes. These should not be exported as arrays but seems like the abc exporter does that. Again, maybe I need to not use the default abc exporter and instead use the alembic api to export data in the required manner.

 

I think you're going to need to export AbcCollections in order to get Sets in Gaffer. I don't know if/how to export AbcCollections from Maya though. One would hope Maya Sets just get exported as Collections by default, but maybe that is not the case?

The maya abc exporter does not export maya object sets so that'll also need to be exported via the api.

Again, thanks for taking a look! I'll keep debugging this and share my findings.

Cheers,

Sachin

[Sachin Shrestha]

It turns out that the maya alembic exporter UI actually does not expose the user attributes. Instead it sends the attribute / attribute prefixes specified by the user in the UI as geometry attributes and hence it ends up writing them as arbGeomParams instead of userProperties. However, if use the plugin command AbcExport(), then I can pass the user attribute names with the -userAttr flag and they export as userProperties and they load correctly in gaffer as well. I'm yet to figure out the primvar data type discrepancy.

I guess until the AbcCollection() to Gaffer.Set() workflow is available, I could use these attributes to auto-create sets in gaffer.

Cheers,

Sachin

[Andrew Kaufman]

Great, glad you got one step further with it. Hopefully we'll get to the collection translation soon.

Cheers,

Andrew

PS. It turns out the Anatomy of a Scene article went live with Gaffer 0.50

--