Maya alembic plugin + referencing: problems and proposed solution

[Paul Molodowitch]

Hi all - so, while we've been using alembic here at Luma for a while, and have been generally quite pleased with it; one area that's always created headaches, though, is the use of alembic nodes and references.

NOTE: throughout here, I'll be talking about about "meshes" for simplicity, but the same principles can be applied to all the shape types that alembic supports - ie, replace "outPolyMesh" with "outNSurface", etc...

The Issue

In maya, meshes originate as numbered indices on AlembicNode.outPolyMesh, which are then connected to normal mesh nodes:

myAlembicNode.outPolyMesh[12] => trogdorShape.inMesh

The problem is that many maya operations will try to insert themselves in this chain.  For instance, if you try to do per-face-assignment, you get:

myAlembicNode.outPolyMesh[12] => groupParts1.inputGeometry

groupParts3.outputGeometry => trogdorShape.inMesh

 

...and if myAlembicNode is inside of a reference, these reference edits are created:

disconnectAttr("myAlembicNode.outPolyMesh[12]", "trogdorShape.inMesh")

connectAttr("myAlembicNode.outPolyMesh[12]", "groupParts1.inputGeometry")

connectAttr("groupParts3.outputGeometry", "trogdorShape.inMesh")

So far, so good... the problem is that if the alembic file is updated - for instance, if a completely new mesh is added - there is no guarantee that "trogdorShape" will still map to outPolyMesh[12] - it might become outPolyMesh[29] after.  So maya's referencing system will now effectively break - it may connect the wrong input mesh into trogdorShape.inMesh, or it may connect nothing at all.

The Proposed Solution

You could try to solve this by some sort of solution which automatically remaps connections when an alembic node is updated, but this approach is sort of fragile, and is sort of trying to fight the way that maya referencing wants to work.

In my opinion, the best solution is one which makes the attribute names much more tightly tied to the underlying shape names, so that if the shape name hasn't changed, the attribute name won't either.  Ideally, we could create attribute names that directly mapped in an easy one-to-one way to the underlyingShape names - ie:

myAlembicNode.outPolyMeshByName.trogdorShape => trogdorShape.inMesh

Unfortunately, this doesn't work, because the shape names can have hierarchies, and attribute names on maya nodes must be unique across the whole node (not just unique amongst it's parent attribute's children).   Furthermore, alembic node names allow a greater range of characters than maya attribute names.

So, since we can't make the attribute names match directly, the next best solution would be to generate the attribute name based on a hash of the shape name - something like:

myAlembicNode.outPolyMeshByHash._138be907f2[0] => trogdorShape.inMesh

The idea is that we would have a guarantee that "trogdorShape" would always map to 138be907f2, so as long as it's name didn't change, it's attribute wouldn't either... unless there was a hash collision.  We should choose the length of the hash such that hash collisions should be EXTREMELY unlikely... but we can also make the hash-attribute-name an array node to deal with collisions, if they do happen (thus the [0] in the above example).  Or, we could just say that hash collisions are sufficiently unlikely that we can ignore the possibility, and not bother with making it an array.

In any case, this would solve the issue with alembic nodes breaking under maya referencing.  For backwards compatibility, we could make it so that both outPolyMesh and outPolyMeshByHash exist, and are valid... but that newly created nodes would make their connections using the named hash attributes by default.

Querying Mesh Names

Additionally, it would be nice if there were a way to query the AlembicNode directly about what the source mesh name was for a given output, to help fix things up in case something goes wrong. Simple read-only attributes on the AlembicNode would help - ie, we could create an array of string names corresponding to the indices of outPolyMesh:

myAlembicNode.outPolyMeshNames[12]: "world/trogdor/trogdorShape"

Or, if we go with a new hash-based attribute naming scheme, we could make the new attributes compound attrs:

myAlembicNode.polyMeshByHash._138be907f2[0].outMesh => trogdorShape.inMesh

myAlembicNode.polyMeshByHash._138be907f2[0].name: "

world/trogdor/trogdorShape"

myAlembicNode.polyMeshByHash._138be907f2[0].index: 12  (for tying to the "old" outPolyMesh attribute)

So - any thoughts on all this? Anybody else have similar problems, or better solutions? Is this something the maintainers might address in a future release? Or at least would endorse?

- Paul

 

[Eoin Murphy]

Hi Paul,

one proprietry geo cache format I've worked with tried to solve this problem by storing a table of in order mesh names as an attribute on the maya node, generated when the node was created at import time (not that different from what you're suggesting at the bottom of your post). If the file path on the node was updated, it would trigger a callback which would generate a mapping table of old/new mesh indices, allowing index based referencing to continue working. It wasn't incredibly efficient in that it had to generate the table in memory each time the scene was opened.. but it did work. It was also a little bit easier as the name scoping rules more or less guaranteed name-uniqueness per cache file.  There was also a patch mechanism which queried the new file and the old list and rewired the connections between the geo cache node and the maya geometry so they matched.

Eoin

[Alexander Kalyuzhnyy]

Hi
As general rule we use it right now, we not working with meshes directly, but rather connect it to another shape and work with that. we use fact that shape names generated by AbcImport are stable.

But we not so heavy users, we use AbcImport only for char FX work, we dont assign materials directly to reference meshes ( we use other inhouse systems for that)

I thought of two other solutions to problem:

1) We can do same thing, as maya do with blendshapes: we can just alias attriubute outPolyMeshNames[12] to something like world_trogdor_trogdorShape (dont forget about symbol escaping). Nice things about this solution: easy to implement and we can alias all output types, since fullpath to abc_node always different.

2) Another solution may resemble Exocortex Alembic, or houdini abc nodes. I mean node that loads geometry or transform data, by fullpath. This way we have more stable, and flexible abc_import tools, but number of nodes will explode as you may guess.

But that causes another technical problem, that you need global hold alembic_cache for ALL nodes,  each node must share file handles, and  pr this thing will be slower than traditional AbcImport node. It must be static cache, or maybe even better transferred as custom MPxData between nodes.

cheers

вторник, 2 декабря 2014 г., 1:54:29 UTC+3 пользователь Paul Molodowitch написал:

[Paul Molodowitch]

On Fri, Dec 5, 2014 at 7:02 PM, Eoin Murphy <eoin....@gmail.com> wrote:

Hi Paul,

one proprietry geo cache format I've worked with tried to solve this problem by storing a table of in order mesh names as an attribute on the maya node, generated when the node was created at import time (not that different from what you're suggesting at the bottom of your post). If the file path on the node was updated, it would trigger a callback which would generate a mapping table of old/new mesh indices, allowing index based referencing to continue working. It wasn't incredibly efficient in that it had to generate the table in memory each time the scene was opened.. but it did work. It was also a little bit easier as the name scoping rules more or less guaranteed name-uniqueness per cache file.  There was also a patch mechanism which queried the new file and the old list and rewired the connections between the geo cache node and the maya geometry so they matched.

Eoin

 

This WOULD work, but it seems to me more of a work-around the fundamental problem... and likely more fragile as a result.  This basic approach could be used to create a work-around as a third-party, though...

 

1) We can do same thing, as maya do with blendshapes: we can just alias attriubute outPolyMeshNames[12] to something like world_trogdor_trogdorShape (dont forget about symbol escaping). Nice things about this solution: easy to implement and we can alias all output types, since fullpath to abc_node always different.

This is basically what I first suggest - the problem is that there's no easy way to map alembic node names to maya attribute names that will guarantee that we avoid naming collisions, since alembic node names are much more permissive. Thus my suggestion of using "hashed" names.

 

2) Another solution may resemble Exocortex Alembic, or houdini abc nodes. I mean node that loads geometry or transform data, by fullpath. This way we have more stable, and flexible abc_import tools, but number of nodes will explode as you may guess.

But that causes another technical problem, that you need global hold alembic_cache for ALL nodes,  each node must share file handles, and  pr this thing will be slower than traditional AbcImport node. It must be static cache, or maybe even better transferred as custom MPxData between nodes.

This would probably help in a lot of situations - and it might be a nice feature to have in general.  But, as you point out, it could get unwieldy in a scene with a lot of meshes.  It also likely harder to implement.  But it could solve our issues, if it was reasonably efficient...

- Paul