Support for arbitrary objects ?

[Martin Tepper]

Hello !

I need to serialize and deserialize big graphs of objects really
quickly. The catch is that the objects are instances of custom Ruby
classes. Is this possible with MessagePack ? My first tries were
negative, but it doesn't say anywhere what MsgPack can't do...

Thanks,
Martin

[amr537]

Hi !

I need to serialize / deserialize (in Python) my own objects too. I wonder if there are an interface (or functions) we can implement that makes an object seralizable.

It could be nice to have something like this:

class MyObj():

  def __init__(self):

    self.value1 = 0

    self.value2 = 0

  def serialize(self): # or packb(self, MyObj)

    return msgpack.packb( [self.value1, self.value2] )

a = MyObj()

MyList = []

MyList.append(a)

buffer = msgpack.packb(MyList)

and the deserialize method too.

[amr537]

If there ara an already implemented way.... please tell....

[amr537]

Hi, I found a way to achieve this.
It can be made by implementing a sort of interface (abstract class)
that defines a getTuple and setTuple methods.

class iSerializable(object):
"""Serializable Object with a Tuple representation of data"""

def getTuple(self):
"""Return a Tuple that represents the data of the object"""
raise NotImplementedError( "Should have implemented
iSerializable.getTuple()" )

def setTuple(self, aDataTuple):
"""Set all data values of the object from a Tuple
representation"""
raise NotImplementedError( "Should have implemented
iSerializable.setTuple(aDataTuple)" )

Then, the above example will be:

class MyObj(iSerializable):

def __init__(self):
self.value1 = 0
self.value2 = 0

def getTuple(self):
return (self.value1, self.value2)

def setTuple(self, aDataTuple):
self.value1 = aDataTuple[0]
self.value2 = aDataTuple[1]

class MyObjList(iSerializable):
def __init__(self):
self.lst = []
def append(self, aMyObj):
self.lst.append(aMyObj)
def getTuple(self):
res = ()
res = tuple([o.getTuple() for o in self.lst])
return res
def setTuple(self, aDataTuple):
self.lst = []
for t in aDataTuple:
a = MyObj()
a.setTuple(t)
self.lst.append(a)

MyList = MyObjList()
a = MyObj()
a.value1 = 11
a.value2 = 12
MyList.append(a)
a = MyObj()
a.value1 = 21
a.value2 = 22
MyList.append(a)

buffer = msgpack.packb(MyList.getTuple())
print buffer
MyOtherList = MyObjList()
MyOtherList.setTuple(msgpack.unpackb(buffer))

Enjoy.

al

[mark.schloesser]

On 24 Dez., 02:57, amr537 <amr...@gmail.com> wrote:
> Hi !
>
> I need to serialize / deserialize (in Python) my own objects too. I wonder
> if there are an interface (or functions) we can implement that makes an
> object seralizable.
>

You can do this by giving pack/packb the "default" argument. This
should
be a function that serializes your custom objects. I suggested a
different API to the Python binding maintainer but he ignored it back
then. In my opinion something like a custom Packer/Unpacker or even
something like you suggest (checking for a serialize function and then
calling it) would be a more intuitive and cool way to handle this.

Well at least it is possible giving a function to pack through the
"default" parameter. This API is inspired by simplejson/json where you
would give the dump function a custom encoder that has a default()
method implemented.

I maintain a github fork of msgpack. I just added the support for your
serialize method (it needs to be _serialize).

The deserialize part is not so straightforward, as you would need to
register those with msgpack for it being able to find the deserialize
methods. So for this i suggest you use the custom unpacker API that I
got.

Check out
https://github.com/rep/msgpack

I attached an example on how it works.

Cheers,
-Mark

import msgpack

class foo(object):
def __init__(self, x):
self.x = x
def _serialize(self):
print 'called'
return ('foo', self.x)

a = foo(5)
dumped = msgpack.packb(a)
print 'after packing', repr(dumped)

class CustomUnpacker(msgpack.Unpacker):
def array_cb(self, o):
print 'array_cb called with', o
if len(o) == 2 and o[0] == 'foo':
return foo(o[1])

cu = CustomUnpacker()
cu.feed(dumped)
loaded = cu.unpack()
print 'after unpacking', loaded

[mark.schloesser]

On 24 Dez., 20:19, amr537 <amr...@gmail.com> wrote:
> Hi, I found a way to achieve this.
> It can be made by implementing a sort of interface (abstract class)
> that defines a getTuple and setTuple methods.
>

[..]

> buffer = msgpack.packb(MyList.getTuple())
> print buffer
> MyOtherList = MyObjList()
> MyOtherList.setTuple(msgpack.unpackb(buffer))
>
> Enjoy.

This is not a solution - you just serialize another object. Apparently
half-way you forgot what you originally wanted to do. What you need is
that msgpack supports your custom class - so that you can serialize
arbitrary object graphs that contain some instances of your class.

E.g. msgpack.packb({'foo':['bar', 4, Myobj()], 'dead': 'beef'})

How would such a thing be possible with defining getTuple and setTuple
methods and calling them before giving anything to msgpack? That is
just - ahm - well ...