Is there a way of adding dict as an extra attribute to objects?

860 views
Skip to first unread message

Benjam901

unread,
Mar 2, 2018, 10:14:11 AM3/2/18
to Python Programming for Autodesk Maya
Hello all,

I was wondering if anyone had a way of adding a dict of data to a maya object as an extra attribute.

I have a scene in which all objects under a node need some data tracked to each, so I am going to add an attribute to the root node and list all objects and values on that top node.

For example:
data = { obj1:cube, obj2:cube, obj3:sphere }

The only way I can find to do this right now is to convert my dict to a string and then back again when I want to read it.

Is there a way to actually add the dict type as an extra attribute?

Cheers,

Ben

AK Eric

unread,
Mar 2, 2018, 11:51:42 AM3/2/18
to Python Programming for Autodesk Maya
Indeed there is:  Python comes with the cPickle module, that lets you serialize any builtin Python data to disk.  It has a dump method that does that writing to your filesystem, but it also had a dumps (dump string) that writes it as a string.

In Maya, you can make string attrs on your nodes...

pseudo code, not directly tested but you get the idea

import cPickle

data = {"a":1, "b":2}
stringData = cPickle.dumps(data)

# Add a custom string attr to your node and:
mc.setAttr("myObj.stringAttr", stringData, type='string')

# Then later to query it:
stringAttrData = str(mc.getAttr("myObj.stringAttr"))
loadedData = cPickle.loads(stringAttrData)

print loadedData
{"a":1, "b":2}

I do this all the time to just get away from having to deal with Maya's own attr types.  So much more flexibility in data storage on Maya nodes.




Marcus Ottosson

unread,
Mar 2, 2018, 11:52:34 AM3/2/18
to python_in...@googlegroups.com

No, there isn’t a datatype for dictionaries.

Converting to string via json.dumps() is one way, you can also convert to a more compact string via something like cPickle.dumps(my_dict), or compress it to e.g. zip and then dumps it, but as far as Maya is concerned they are all of type string. Odds are you can convert it into any of the other available types, like a huge number, but string isn’t a bad option; it’s got no size limit (other than your total amount of RAM and disk space) and probably isn’t posing any relevant performance overhead on its own in terms of reading and writing to the attribute. The conversion between Maya and Python can however be costly, and pickle is likely the faster option in this case (depending on your data). The only thing to look out for with that is that it will limit the result to machine-readable output (e.g. Xz\939150\0klk5) and furthermore only be readable by the particular version of Python you are running at the time of performing the dump. That is, if you store a dumped pickle in Maya 2013 on Linux, odds are you can’t read it in 2018 on Windows (or even the same version of Maya but on different OSes) etc. On the other hand, you can pickle more than just dictionaries. Whole classes and instanced PyMel nodes and what else have you.

You can see what Maya types are available as extra (i.e. “dynamic”) attributes here for example.

Justin Israel

unread,
Mar 2, 2018, 3:54:31 PM3/2/18
to python_in...@googlegroups.com


On Sat, Mar 3, 2018, 5:52 AM Marcus Ottosson <konstr...@gmail.com> wrote:

No, there isn’t a datatype for dictionaries.

Converting to string via json.dumps() is one way, you can also convert to a more compact string via something like cPickle.dumps(my_dict), or compress it to e.g. zip and then dumps it, but as far as Maya is concerned they are all of type string. Odds are you can convert it into any of the other available types, like a huge number, but string isn’t a bad option; it’s got no size limit (other than your total amount of RAM and disk space) and probably isn’t posing any relevant performance overhead on its own in terms of reading and writing to the attribute. The conversion between Maya and Python can however be costly, and pickle is likely the faster option in this case (depending on your data). The only thing to look out for with that is that it will limit the result to machine-readable output (e.g. Xz\939150\0klk5) and furthermore only be readable by the particular version of Python you are running at the time of performing the dump. That is, if you store a dumped pickle in Maya 2013 on Linux, odds are you can’t read it in 2018 on Windows (or even the same version of Maya but on different OSes) etc.

Why would that be true? If you specify the protocol when you read and write the cPickle data, then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows. 


One could stick with binary protocol 2 the whole way through, I would think. Am I wrong? 


On the other hand, you can pickle more than just dictionaries. Whole classes and instanced PyMel nodes and what else have you.

You can see what Maya types are available as extra (i.e. “dynamic”) attributes here for example.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOA3GN9tH%3Dz13BwwVQo35fzkv63-c4iM3xuuSsmZxJ5f%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Marcus Ottosson

unread,
Mar 3, 2018, 5:00:03 AM3/3/18
to python_in...@googlegroups.com

then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows.

TLDR; compiler differences and unavailable internal datastructures.

For starters, one thing that could be confusing is how there is pickle and then there is cPickle. pickle is a pure Python library which I expect have no trouble with cross-platform compatibility. cPickle on the other hand is a compiled library and, like anything compiled, I’d expect the same incompatibility as with it as with e.g. PyQt unless it’s compiled for the particular version of Maya’s Python interpreter, and the same for NumPy and friends. (Can this be not true?)

However aside from that, with both pickle and cPickle you are still serialising internal data structures and not all datastructures on Linux are available on Windows, not all in Maya 2013 are available in Maya 2018. In the simplest case of incompatibility, you could pickle a PyMEL object in Maya 2018 that hasn’t yet been invented in 2013, or a MASH object that didn’t exist in 2013. Or pickle some instance of one of your own plug-ins, that you’ve got a different version of on another computer.

I would think that the only way to be safe in a multi-OS, multi-Maya environment, is to both stick with pickle (not cPickle) and ensure your data is JSON-serialisable to begin with (no complex data). But in that case, you’ve run out of reasons to use pickle over json anyway.

Taking a closer look at the documentation, I also spotted..

The pickle serialization format is guaranteed to be backwards compatible across Python releases.

Which I would read “data pickled in Maya 2013 is guaranteed to be compatible with Maya 2018”, but not the other way around (that would be forwards compatibility). This, I think, would invalidate even the “safe” option I listed just now.

With all this said, I did expect to find larger warning signs explicitly written out in both the Python docs and community, but can’t seem to find a conclusive answer. So with that in mind, I’ll change my position from “Odds are you will run into trouble” to “Odds are you may run into trouble”. Would you agree?


On 2 March 2018 at 20:54, Justin Israel <justin...@gmail.com> wrote:


On Sat, Mar 3, 2018, 5:52 AM Marcus Ottosson <konstr...@gmail.com> wrote:

No, there isn’t a datatype for dictionaries.

Converting to string via json.dumps() is one way, you can also convert to a more compact string via something like cPickle.dumps(my_dict), or compress it to e.g. zip and then dumps it, but as far as Maya is concerned they are all of type string. Odds are you can convert it into any of the other available types, like a huge number, but string isn’t a bad option; it’s got no size limit (other than your total amount of RAM and disk space) and probably isn’t posing any relevant performance overhead on its own in terms of reading and writing to the attribute. The conversion between Maya and Python can however be costly, and pickle is likely the faster option in this case (depending on your data). The only thing to look out for with that is that it will limit the result to machine-readable output (e.g. Xz\939150\0klk5) and furthermore only be readable by the particular version of Python you are running at the time of performing the dump. That is, if you store a dumped pickle in Maya 2013 on Linux, odds are you can’t read it in 2018 on Windows (or even the same version of Maya but on different OSes) etc.

Why would that be true? If you specify the protocol when you read and write the cPickle data, then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows. 


One could stick with binary protocol 2 the whole way through, I would think. Am I wrong? 


On the other hand, you can pickle more than just dictionaries. Whole classes and instanced PyMel nodes and what else have you.

You can see what Maya types are available as extra (i.e. “dynamic”) attributes here for example.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA38%2BW2k7W1bpMYsnnincG9S%2BgS2uW9Fknh5rCpnYHuFtA%40mail.gmail.com.

Kenneth Ibrahim

unread,
Mar 3, 2018, 1:08:02 PM3/3/18
to python_in...@googlegroups.com
Could you also assign the serialized dictionary to blind data on nodes? I can't remember of the top of my head which data types are supported.

On Sat, Mar 3, 2018 at 1:59 AM, Marcus Ottosson <konstr...@gmail.com> wrote:

then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows.

TLDR; compiler differences and unavailable internal datastructures.

For starters, one thing that could be confusing is how there is pickle and then there is cPickle. pickle is a pure Python library which I expect have no trouble with cross-platform compatibility. cPickle on the other hand is a compiled library and, like anything compiled, I’d expect the same incompatibility as with it as with e.g. PyQt unless it’s compiled for the particular version of Maya’s Python interpreter, and the same for NumPy and friends. (Can this be not true?)

However aside from that, with both pickle and cPickle you are still serialising internal data structures and not all datastructures on Linux are available on Windows, not all in Maya 2013 are available in Maya 2018. In the simplest case of incompatibility, you could pickle a PyMEL object in Maya 2018 that hasn’t yet been invented in 2013, or a MASH object that didn’t exist in 2013. Or pickle some instance of one of your own plug-ins, that you’ve got a different version of on another computer.

I would think that the only way to be safe in a multi-OS, multi-Maya environment, is to both stick with pickle (not cPickle) and ensure your data is JSON-serialisable to begin with (no complex data). But in that case, you’ve run out of reasons to use pickle over json anyway.

Taking a closer look at the documentation, I also spotted..

The pickle serialization format is guaranteed to be backwards compatible across Python releases.

Which I would read “data pickled in Maya 2013 is guaranteed to be compatible with Maya 2018”, but not the other way around (that would be forwards compatibility). This, I think, would invalidate even the “safe” option I listed just now.

With all this said, I did expect to find larger warning signs explicitly written out in both the Python docs and community, but can’t seem to find a conclusive answer. So with that in mind, I’ll change my position from “Odds are you will run into trouble” to “Odds are you may run into trouble”. Would you agree?

On 2 March 2018 at 20:54, Justin Israel <justin...@gmail.com> wrote:


On Sat, Mar 3, 2018, 5:52 AM Marcus Ottosson <konstr...@gmail.com> wrote:

No, there isn’t a datatype for dictionaries.

Converting to string via json.dumps() is one way, you can also convert to a more compact string via something like cPickle.dumps(my_dict), or compress it to e.g. zip and then dumps it, but as far as Maya is concerned they are all of type string. Odds are you can convert it into any of the other available types, like a huge number, but string isn’t a bad option; it’s got no size limit (other than your total amount of RAM and disk space) and probably isn’t posing any relevant performance overhead on its own in terms of reading and writing to the attribute. The conversion between Maya and Python can however be costly, and pickle is likely the faster option in this case (depending on your data). The only thing to look out for with that is that it will limit the result to machine-readable output (e.g. Xz\939150\0klk5) and furthermore only be readable by the particular version of Python you are running at the time of performing the dump. That is, if you store a dumped pickle in Maya 2013 on Linux, odds are you can’t read it in 2018 on Windows (or even the same version of Maya but on different OSes) etc.

Why would that be true? If you specify the protocol when you read and write the cPickle data, then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows. 


One could stick with binary protocol 2 the whole way through, I would think. Am I wrong? 


On the other hand, you can pickle more than just dictionaries. Whole classes and instanced PyMel nodes and what else have you.

You can see what Maya types are available as extra (i.e. “dynamic”) attributes here for example.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
"God is a metaphor for a mystery that absolutely transcends all human categories of thought. It's as simple as that!" - Joseph Campbell

Justin Israel

unread,
Mar 3, 2018, 2:38:50 PM3/3/18
to python_in...@googlegroups.com


On Sat, Mar 3, 2018, 10:59 PM Marcus Ottosson <konstr...@gmail.com> wrote:

then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows.

TLDR; compiler differences and unavailable internal datastructures.

For starters, one thing that could be confusing is how there is pickle and then there is cPickle. pickle is a pure Python library which I expect have no trouble with cross-platform compatibility. cPickle on the other hand is a compiled library and, like anything compiled, I’d expect the same incompatibility as with it as with e.g. PyQt unless it’s compiled for the particular version of Maya’s Python interpreter, and the same for NumPy and friends. (Can this be not true?)

However aside from that, with both pickle and cPickle you are still serialising internal data structures and not all datastructures on Linux are available on Windows, not all in Maya 2013 are available in Maya 2018. In the simplest case of incompatibility, you could pickle a PyMEL object in Maya 2018 that hasn’t yet been invented in 2013, or a MASH object that didn’t exist in 2013. Or pickle some instance of one of your own plug-ins, that you’ve got a different version of on another computer.

I would think that the only way to be safe in a multi-OS, multi-Maya environment, is to both stick with pickle (not cPickle) and ensure your data is JSON-serialisable to begin with (no complex data). But in that case, you’ve run out of reasons to use pickle over json anyway.

Taking a closer look at the documentation, I also spotted..

The pickle serialization format is guaranteed to be backwards compatible across Python releases.

Which I would read “data pickled in Maya 2013 is guaranteed to be compatible with Maya 2018”, but not the other way around (that would be forwards compatibility). This, I think, would invalidate even the “safe” option I listed just now.

With all this said, I did expect to find larger warning signs explicitly written out in both the Python docs and community, but can’t seem to find a conclusive answer. So with that in mind, I’ll change my position from “Odds are you will run into trouble” to “Odds are you may run into trouble”. Would you agree?


The only part of all those points that I agree with is that you would have trouble if you serialised a custom object that was not available on a subsequent load of that data. It could happen for reasons beyond just two versions of Maya. One install could be missing a plugin or library that the original used. But that can be avoided if one just makes sure to serialise only basic types, or types that are part of the same program that is doing the loading and saving. I would like to believe that this data is part of some program, which has some logic around reading and writing the data. This program would use the data consistently regardless of Maya 2013 or 2018. You couldnt expect a program to always work correctly if some external code was modifying the custom data outside the spec of the program. 

On your other points... Pickle vs cPickle is just a compiled implementation vs pure python, for speed ups. The protocol is the same. And no one is manually building and installing cPickle because it is part of the CPython stdlib. If you are worried about Maya 2020 having some new python implementation that no longer ships with cPickle, then the program can just do

try: 
    import cPickle as pickle
except ImportError: 
    import pickle 


And regarding the serialised format compatibility, if you specify the protocol number then you use the same format regardless of future versions. Use protocol 2 now, and continue using it in your program even when Maya 2020 ships with python 3 and cPickle has a protocol 3 available (not real). 

It would not be expected that random programs would be reading and changing this serialised data if it is in a binary format. It would need to be specific to a python program that that needs it for future runs. If this data is meant to be used by anyone, such as a metadata system, then use a plaintext human readable format like json. But it depends on the purpose for this serialisation. 


On 2 March 2018 at 20:54, Justin Israel <justin...@gmail.com> wrote:


On Sat, Mar 3, 2018, 5:52 AM Marcus Ottosson <konstr...@gmail.com> wrote:

No, there isn’t a datatype for dictionaries.

Converting to string via json.dumps() is one way, you can also convert to a more compact string via something like cPickle.dumps(my_dict), or compress it to e.g. zip and then dumps it, but as far as Maya is concerned they are all of type string. Odds are you can convert it into any of the other available types, like a huge number, but string isn’t a bad option; it’s got no size limit (other than your total amount of RAM and disk space) and probably isn’t posing any relevant performance overhead on its own in terms of reading and writing to the attribute. The conversion between Maya and Python can however be costly, and pickle is likely the faster option in this case (depending on your data). The only thing to look out for with that is that it will limit the result to machine-readable output (e.g. Xz\939150\0klk5) and furthermore only be readable by the particular version of Python you are running at the time of performing the dump. That is, if you store a dumped pickle in Maya 2013 on Linux, odds are you can’t read it in 2018 on Windows (or even the same version of Maya but on different OSes) etc.

Why would that be true? If you specify the protocol when you read and write the cPickle data, then it should be able to be dumped on Maya 2013 in Linux and read in Maya 2018 on Windows. 


One could stick with binary protocol 2 the whole way through, I would think. Am I wrong? 


On the other hand, you can pickle more than just dictionaries. Whole classes and instanced PyMel nodes and what else have you.

You can see what Maya types are available as extra (i.e. “dynamic”) attributes here for example.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODAvy8To4jCAk-n%3DvffzPFKGLiQu%2B19RCgapnLG%2BF%3DBKw%40mail.gmail.com.

Marcus Ottosson

unread,
Mar 4, 2018, 5:36:10 AM3/4/18
to python_in...@googlegroups.com

Yes, that would make sense. It’d be like how mayaAscii and mayaBinary are plain-text and binary respectively, but still equally compatible with Maya, regardless of platform.

Perhaps it’s then safe to surmise that because pickle is spelled out as being cross-platform compatible, and cPickle is compatible with anything coming out of pickle, then anything coming out of cPickle is also cross-platform compatible. Especially when considering this line.

The data streams the two modules produce are guaranteed to be interchangeable.

To the original question, I had another idea that might prove useful.

The goal is to embed a dictionary - i.e. nested - datastructure in Maya, and one way of doing so is to take a Python dictionary, convert it to a string and store it in a Maya string plug.

However, what if you were to consider Maya’s plugs as key/value pairs, where the attribute name is the key, and its value is.. well, the value? In that case, you could either:

  1. Use regular attributes, of varying types, and query them like normal, e.g. cmds.getAttr("myNode.key")
  2. Use a compound attribute, to mimic a Python dictionary, such that the value returned from cmds.getAttr("myNode.myCompound") is similar to what you would normally get out of a dictionary.
data = {
  "project": "Avengers",
  "shot": "0050",
  "task": "animation",
  "started": True
}

cmds.getAttr("myNode.myCompound.project") == u"Avengers"
cmds.getAttr("myNode.myCompound.shot") == u"0050"
cmds.getAttr("myNode.myCompound.started") == True

It’d even get you all attributes at once if you ask nicely.

cmds.getAttr("myNode.myCompound") == (u"Avengers", u"0050", u"animation", True)

However probably not in the order of your original dictionary, and without the actual keys. That said, the order is reliable, as it would be in the order that the attributes were added, so you may be able to determine the key separate from your query. Note that this would also work with nested compound attributes, and more datatypes than would otherwise be supported by JSON, e.g. long, double3, mesh and nurbsCurve data.

So in this case, then Yes, you can embed a dictionary. :)


To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA1hnJpqDSN5AwhRX9jbvgefi5_GnJrf19DQGxW2QgnHDQ%40mail.gmail.com.

Benjam901

unread,
Mar 5, 2018, 8:07:34 AM3/5/18
to Python Programming for Autodesk Maya
This thread has proved extremely informative and useful, thanks for the tips and info on pickle vs cPickle and the various nuances that come with both.

Once again the python maya google group has saved me :)

Cheers,

Ben

Mark Jackson

unread,
Apr 17, 2018, 1:43:05 PM4/17/18
to python_inside_maya
We've always used json to convert dicts into a string for serializing onto a node and never had any issue. The only caveat that we've found over the years, and I'm not even sure if this is still an issue, is the actual string length you can serialize has to be managed. When we wrote a lot of our base code if the attr was unlocked them Maya would truncate it when the node was loaded, if it were locked it didn't act the same.

    def __serializeComplex(self, data):
        '''
        Serialize complex data such as dicts to a JSON string
        '''
        if len(data) > 32700:
            log.debug('Warning >> Length of string is over 16bit Maya Attr Template limit - lock this after setting it!')
        return json.dumps(data)

    def __deserializeComplex(self, data):
        '''
        Deserialize data from a JSON string back to it's original complex data
        '''
        if type(data) == unicode:
            return json.loads(str(data))
        return json.loads(data)

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
-------------------------------------
Mark Jackson
CEO / Technical Director
red9consultancy.com
Reply all
Reply to author
Forward
0 new messages