MFnMesh: Creating Color Sets/Setting Vert Colors

[John Patrick]

Hey all,

Was wondering if anyone has experience creating color sets inside of a node that outputs new mesh data. The node I'm working on is an Alembic Geo Cache reader, forked from the Alembic project, which reads an Alembic cache file and creates polyMesh outputs (among other things).

My understanding (which may be totally inaccurate) is that color sets are part of the Mesh data, and therefore I could create them and set vert colors per set like so:

#inside compute

mobj = outHandle.data()

MFnMesh fnMesh = MFnMesh(obj)

fnMesh.create(numVers, numPolys, pointList, polyCounts, polyCncts, theParent) #create the mesh from data in the cache

fnMesh.createColorSetWithName("myAwesomeColorSet")

fnMesh.setFaceVertexColor(50, 0) 

outHandle.set(mobj)

When I do this though, I don't see any colors or color sets on a shape connected to the output attribute on the node (though the mesh shape itself is there).

Can anyone shed some light on this for me?

Best,

JP

--
John Patrick
404-242-2675
jspa...@gmail.com
http://www.canyourigit.com

[John Patrick]

For that matter, does anyone know of any code examples (preferably c++, but Python is fine too) of setting poly vertex colors on an output DataHandle?

[Viktoras]

On 2011.07.22 02:35, John Patrick wrote:

For that matter, does anyone know of any code examples (preferably c++, but Python is fine too) of setting poly vertex colors on an output DataHandle?

actually, I just use MFnMesh.setVertexColors(MColorArray,MIntArray). Tried fiddling around with color sets, but just couldn't find a right way to use them.
One thing to note, there's nasty bug (confirmed by Autodesk) in 2010 maya that spams output with some warning eachtime using setVertexColor /setVertexColors, so a better choice would be to use setVertexColors() to minimize that spam.

all in all, i have a custom node that takes a mesh as an input, then using some custom plugin internal data figures out colors, and returns an output mesh that is identical to input except that it changes vertex colors. relevant code would be..

    MDataHandle inMeshData = dataBlock.inputValue(ngLayerColorDisplayNode::attrInMesh);
    MDataHandle outMeshData = dataBlock.outputValue(ngLayerColorDisplayNode::attrOutMesh);
   
    outMeshData.copy(inMeshData);

//.. skipping code for calculating vertex colors into MColorArray
   
    MFnMesh meshFn(outMeshData.asMesh());
    meshFn.setVertexColors(this->currColors,this->vertexIndexes);
    dataBlock.setClean(plug);

[John Patrick]

Thanks Viktoras, I'll give setting the colors that way a shot.

As a side note, do you know what the difference is between:

 

MFnMesh meshFn(outMeshData.asMesh())  vs  MFnMesh meshFn(outMeshData.asData())?

Both asData() and asMesh ()return an MObject, but I'm wondering if that MObject is pointing to different data?

Anyways, once I get into work I'll give it a shot and report back.

--
view archives: http://groups.google.com/group/python_inside_maya
change your subscription settings: http://groups.google.com/group/python_inside_maya/subscribe

[John Patrick]

Okay, so good news/bad news.  I can set colors on whatever the current color set is.  In fact, I was actually doing it the whole time, I just didn't have the display vertex colors option turned on for my shape (doh!!).

The color sets thing is a real problem though.  There must be a way to do this...

[Viktoras]

On 2011.07.22 18:16, John Patrick wrote:
> MFnMesh meshFn(outMeshData.asMesh()) vs MFnMesh
> meshFn(outMeshData.asData())?
>
> Both asData() and asMesh ()return an MObject, but I'm wondering if
> that MObject is pointing to different data?
>

you mean .data(), as in MDataHandle.data() ? That method is for
retreiving handle values for custom data types, the ones you can create
via MPxData.

[John Patrick]

Thanks Viktoras (sorry for the delay).

I just wanted to report back that there were two tricks to creating color sets and setting colors on those color sets.

The first is that if you have an MFnMesh with an MObject from an attribute, you can't create a color set using the normal method.  Instead, there is an undocumented method in the MFnMesh header file called "createColorSetDataMesh(setName)".

The second trick (or realization) was that you can't actually "set" a color set from in the node's compute method, at least not when the MFnMesh's MObject is an attribute.  Most of the setColor methods set colors on the *current* color set, but setColors and assignColors both take an optional color set arg.  So to set the colors for myColorSet:

meshFn.setColors(MColorArray of colors in per-face, per-vertex order, colorSetName)

meshFn.assignColors(colorIndexArray)

the colorIndexArray is an MIntArray of the index in the MColorArray for which each vace-vert will get assigned. In other words, in this case, it just an ordered list from 0-numFaceVerts.  I'm assuming this is for efficiency, so that if you only were using 2 different colors on a mesh, you could do (in pseudo code): 

MFn.setColors(MColorArray(Color1, Color2), myColorSet)

MFn.assignColors(indexArray(0,1,1,1,0...), myColorSet), where indexArray.length() == MFn.numFaceVertices()

So, hopefully that helps if someone happens to Google this later ;)

--
view archives: http://groups.google.com/group/python_inside_maya
change your subscription settings: http://groups.google.com/group/python_inside_maya/subscribe