Newbie Alembic API Questions

[Russ A]

Hi-

I'm trying to develop an Alembic exporter (only!) in C++ that would be part of an existing commercial product that runs on Windows, OS X, and Linux. I got the library itself to compile on each platform after several days work.*

It's been a VERY frustrating process trying to now write code with Alembic, far more so than the other libraries I use. I've been rummaging through code trying to reverse engineer it, but it's just not time-effective for me.  I'm ready to throw in the towel and get back to productive work, but thought I'd ask here to see if I can get somewhere. Although I think it would be nice to support Alembic, I have limited time available to do so.

So here are a few specific questions. Apologies in advance for the length.

* Time index 0 is a fixed timescale starting at 0, stepping by 1. Is that something I should use? Or should I go with a more specific timescale that enumerates specific times and can start at a specific time. What do readers expect?

* If I have a "double xf[16]", what's the simplest way to create a corresponding transformation node? I see OXformSchema, XformOp, and XformSample, but no real explanation. What if it's a constant? What if it varies with time?

* What are the applicable conventions for the node names? Ie the relationship between the name of the mesh node and the transform node? Typically I'm going to expect one of them to be the "real" name, and the other to have "Mesh" or "Transform" appended to it.

* How do I instance a mesh into a whole lot of transforms? I can recreate it a bunch of times, and it sounds like Alembic might recognize the duplication, but that gets tedious and unnecessary after a couple hundred times.

* Is there a nice simple code block to apply an RGB color to a mesh, so that readers will render it with a simple Phong or whatever shader?

* Why do I have to do a getSchema after I create an object, then add sample to the *SCHEMA*, not the object itself?  That's pretty much the opposite to the normal definition of a schema as a *description* of the data.

* What's going on with the various objects during exports: do the objects contain the data in RAM, or are they small references to the data sitting on disk? (seems like the former) The example code seems to emphasize passing things around as references instead of using new, delete, and pointers---the latter permits more flexible coding of dynamic hierarchies. I don't see objects being returned from functions, which would require new/delete/pointers. 

* Any way to get SimpleAbcViewer to attach to a camera in the Alembic file?

Comments are appreciated. 

I've included below some whine-y thoughts and observations from my experience below. I offer them in hopes they might be used to help make Alembic easier to pick up for others. (I know the Alembic developers have limited time too.)

Thanks,

Russ

General thoughts as an Alembic newbie:

* There's way too little documentation. The doxygen-generated stuff is useless, as the code isn't commented at all. 

* There's nothing to tell me what or why I should be doing anything, at most, what a few low-level things do.

* Most of the code and description that is there is concerned with API issues, not about the data going into the file. The API isn't the end product, moving data successfully from one application to another is.

* Alembic and it's documentation shouldn't rely on unstated behaviours/properties/requirements of 3dsmax and maya etc. 

* The coding style seems waaaay more complicated than necessary. I have no idea why there are all these schemas and templates, and thus what is going on under the hood to help me understand how to use it. What problems are they there to solve? If none, they shouldn't be there. 

* I can't help but think that the Alembic API could be rewritten to a much simpler and easier to use form. Filmbox is much simpler, for example. 

* The layered design is formally nice, but the source code is spread across different directories and it's time-consuming to sort through it. Overall there are a few too many layers of hierarchy everywhere, especially ALEMBIC_VERSION_NS ---there are 3 levels of namespace where 1 would suffice).

* Some of the "example" source code is located inside a "bin" directory. 

* There are multiple files with the same name in different directories, an invitation to problems. (main.cpp, IArchive.cpp, OArchive.cpp)

* There need to be small examples that do small useful tasks---create and export an small fixed mesh, a moving mesh, an animated mesh, a camera, ... Filmbox has a lot of little examples like this that are a great help. It's nice to have the code for reference maya/prman/etc importers and exporters,  but insufficient because they conflate the Alembic stuff with that of the other applications, so that you have to reverse-engineer two things, not just one. 

* There need to be some sample files. Since my version of the library is Ogawa-only, I have NO example files. Not one. If I was developing an importer, I would.... not.

* The technique I used to get Alembic and some other open-source libraries to compile lately is to avoid the complex error-prone build processes that are delivered, and create simple flat makefiles, one for each platform. These makefiles depend on one target ("all"), and by adjusting a few variables you can easily retarget them. (You can turn them into scripts easily also.) You wouldn't *develop* this way, but I think open-source libraries would make their users' lives much easier by *delivering* in this fashion.

* Except SimpleAbcViewer on Win. And I have dropped HFS5 support, as this export-only.

[Lucas Miller]

Thanks for the feedback.

So here are a few specific questions. Apologies in advance for the length.

* Time index 0 is a fixed timescale starting at 0, stepping by 1. Is that something I should use? Or should I go with a more specific timescale that enumerates specific times and can start at a specific time. What do readers expect?

Time index 0 is the default time sampling, and it currently corresponds to time starting at 0 seconds with every index mapping to 1 second.

I think the original intention is so the index and time lined up.  If you are sampling your property at 1 frame per 24 seconds, and you were starting at frame 101, you could do something like: TimeSampling(1.0/24.0, 101.0/24.0)

 

* If I have a "double xf[16]", what's the simplest way to create a corresponding transformation node? I see OXformSchema, XformOp, and XformSample, but no real explanation. What if it's a constant? What if it varies with time?

You would create an XformSample with a single XformOp of type matrix.  If it is constant, you would set just 1 value for it, if not, you would set multiple samples, and be sure to set the TimeSampling for it.  If your matrix is identity and doesn't change over time, you don't need to write out anything.

 

* What are the applicable conventions for the node names? Ie the relationship between the name of the mesh node and the transform node? Typically I'm going to expect one of them to be the "real" name, and the other to have "Mesh" or "Transform" appended to it.

The convention typically doesn't matter.  Maya does something like foo for the transform name, fooShape for the mesh name.

 

* How do I instance a mesh into a whole lot of transforms? I can recreate it a bunch of times, and it sounds like Alembic might recognize the duplication, but that gets tedious and unnecessary after a couple hundred times.

OObject::addChildInstance  though many readers just treat it as normal hierarchy instead of IObject::isInstanceDescendant, IObject::isChildInstance, etc.

 

* Is there a nice simple code block to apply an RGB color to a mesh, so that readers will render it with a simple Phong or whatever shader?

Sadly no, various places may do this with a custom arbitrary geometry parameter (a simple example of that could be provided), or defining the material and setting it as part of the material definition.

 

* Why do I have to do a getSchema after I create an object, then add sample to the *SCHEMA*, not the object itself?  That's pretty much the opposite to the normal definition of a schema as a *description* of the data.

The SCHEMA is really a compound property.  The object can technically have multiple schemas, although this is very rare. 

 

* What's going on with the various objects during exports: do the objects contain the data in RAM, or are they small references to the data sitting on disk? (seems like the former) The example code seems to emphasize passing things around as references instead of using new, delete, and pointers---the latter permits more flexible coding of dynamic hierarchies. I don't see objects being returned from functions, which would require new/delete/pointers. 

 

It's more like the former which you mentioned.  The idea is to keep those objects and properties around which you'll need when writing multiple samples, and letting things which you may only need for one sample to go out of scope, to allow much larger hierarchies.

 

* Any way to get SimpleAbcViewer to attach to a camera in the Alembic file?

I think the python example viewer AbcView does this.

 

General thoughts as an Alembic newbie:

* There's way too little documentation. The doxygen-generated stuff is useless, as the code isn't commented at all. 

* There's nothing to tell me what or why I should be doing anything, at most, what a few low-level things do.

* Most of the code and description that is there is concerned with API issues, not about the data going into the file. The API isn't the end product, moving data successfully from one application to another is.

You can help improve this by signing the Individual contributors agreement here:

https://docs.google.com/file/d/0B_S7TABLaH6xanl2X2xsMGowZFk

or the Corporate contributors agreement here:

https://docs.google.com/file/d/0B_S7TABLaH6xYnlSQlJPdXJXVzA

and submitting a guide for inclusion as a github wiki.

* Alembic and it's documentation shouldn't rely on unstated behaviours/properties/requirements of 3dsmax and maya etc. 

It doesn't, it may offer an explanation about where maya or 3dsmax properties may go.

 

* The coding style seems waaaay more complicated than necessary. I have no idea why there are all these schemas and templates, and thus what is going on under the hood to help me understand how to use it. What problems are they there to solve? If none, they shouldn't be there. 

Agreed.  I think properties are appropriately templated to accommodate the various types of data.  Schemas got out of control especially with their constructors.  Refactoring that part is something I've wanted to do for a while.  (while making sure most existing code continue to compile)

* There need to be small examples that do small useful tasks---create and export an small fixed mesh, a moving mesh, an animated mesh, a camera, ... Filmbox has a lot of little examples like this that are a great help.

* There need to be some sample files. Since my version of the library is Ogawa-only, I have NO example files. Not one. If I was developing an importer, I would.... not.

Would this be best provided via wiki or an alternate examples repo?

Lucas

[Russ A]

 

Thanks for the message. I think I've been able to implement the things that Alembic is able to do.

* Is there a nice simple code block to apply an RGB color to a mesh, so that readers will render it with a simple Phong or whatever shader?

Sadly no, various places may do this with a custom arbitrary geometry parameter (a simple example of that could be provided), or defining the material and setting it as part of the material definition.

I think the inability to set a color will be quite unpopular with users... in my case they are dealing with hundreds of objects and frequently use colors to make it visually easier to tell what's what. (If a material can be described so that most shipping Alembic importers will interpret it correctly, please do tell.)

For custom workflows where there is a custom exporter and custom importer at each end, being able to extend Alembic with custom properties is great. But for widely-used shrink-wrap applications that are trying to interchange using Alembic, it is essential to migrate to standardized builtin properties that are simply and widely supported. (Other candidates I've noted: locators, lights, opacity, and show/hide visibility status) Standardized generic properties and objects (a point light! a directional light! a generic spot light!) that are widely supported are more useful than saying "you can do whatever you want in your exporter and your importer", which results in audience of one. The customization should layer over the standardized part, so that off-the-shelf importers (say in C4D, Blender, Modo, stock Maya, etc) can still do SOMETHING useful when they don't understand the custom part.

[For my purposes, being able to specify uvmapped texture maps would be helpful, either with in-file data or via file name, and of course sample by sample if appropriate. If it's been set up in my app, users want it downstream, even if they'll refine the materials/compositing in another app.]

 * Why do I have to do a getSchema after I create an object, then add sample to the *SCHEMA*, not the object itself?  That's pretty much the opposite to the normal definition of a schema as a *description* of the data.

The SCHEMA is really a compound property.  The object can technically have multiple schemas, although this is very rare. 

After describing properties at length in the docs, using a different undocumented misnomer term for it in the API wasn't something I expected. sed s/Schema/Property/g would help me. Similarly a convenience function set(x) => set.getSchema(x) to reduce unnecessary boilerplate.

* Any way to get SimpleAbcViewer to attach to a camera in the Alembic file?

I think the python example viewer AbcView does this.

My perception is that trying to build the python interface including identifying and downloading all the necessary prerequisites would take the better part of the day, just for one platform. It may be easy enough if you already regularly use python, PyQt, and compiled-in python modules, but if not...

It would be incredibly useful to have a precompiled full-function "reference viewer" for primary platforms (ie Windows and Mac) with no dependencies, to use as a gold standard for new importers or exporters, independent of other software that may or may not be complete or accurate or in one's hands (especially since many commercial offerings are incomplete).

 

* Alembic and it's documentation shouldn't rely on unstated behaviours/properties/requirements of 3dsmax and maya etc. 

It doesn't, it may offer an explanation about where maya or 3dsmax properties may go.

The data itself may be platform-agnostic, but implicitly it is dependent because of the lack of documentation --- one can only guess based on what the other software does. Alembic as a transport container is nice, but to be an interchange standard the data content itself must be specified. 

Simple examples: Is Alembic data always Y up? Nothing definitive (max exporter has a selection?). What axis does the camera look down? -Z? Nothing says. I hope so, because that's what I've assumed and I have no way to tell. What is GeometryScope? Can it be used to supply face normals?

* There need to be some sample files. Since my version of the library is Ogawa-only, I have NO example files. Not one. If I was developing an importer, I would.... not.

Would this be best provided via wiki or an alternate examples repo?

 

The download page of the Wiki, where the Octopus is located, would seem to be a good location.

Hopefully my comments can help guide Alembic towards being useful for commercial interchange, in addition to its use in custom workflows.

Thanks,

Russ