Unpickling of model instances fails when using mapped collections

[Alex Grönholm]

I'm having trouble unpickling model instances where the class has an attribute-mapped collection:

import pickle

from sqlalchemy import create_engine, Column, Integer, Unicode, ForeignKey
from sqlalchemy.orm import Session, relationship
from sqlalchemy.orm.collections import attribute_mapped_collection
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()
engine = create_engine('sqlite:///')
session = Session(engine)

class Parent(Base):
    __tablename__ = 'parents'
    id = Column(Integer, primary_key=True)
   
    children = relationship('Child', cascade='all, delete-orphan', lazy='subquery', collection_class=attribute_mapped_collection('key'))

class Child(Base):
    __tablename__ = 'children'
    parent_id = Column(Integer, ForeignKey(Parent.id), primary_key=True)
    key = Column(Integer, primary_key=True)
    value = Column(Unicode)

Base.metadata.create_all(engine)
parent = Parent()
session.add(parent)
session.flush()
session.refresh(parent)
pickled = pickle.dumps(parent, 2)
pickle.loads(pickled)


Executing this results in:

Traceback (most recent call last):
  File "sqlatest.py", line 32, in <module>
    pickle.loads(pickled)
  File "/usr/lib/python2.7/pickle.py", line 1382, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1083, in load_newobj
    obj = cls.__new__(cls, *args)
TypeError: attrgetter expected 1 arguments, got 0


If I remove the relationship, pickling and unpickling work fine. None of the other mapped collection variants work either, though they produce different tracebacks. Is this a bug or am I doing something wrong?

[Alex Grönholm]

So basically this comes down the operator.attrgetter not being serializable. I'm wondering why something like this would be serialized at all -- aren't collections part of the class definition?

[Michael Bayer]

Whoops, forgot to get to this email.

These pickle problems are very tough since the pickle module gives you no indication of how it gets to what can't be serialized.

I see you're using pickle format "2" to try to get around the issue but that's not really a solution here.

A callable that's serializable would need to be used, this can be fixed in SQLAlchemy (added http://www.sqlalchemy.org/trac/ticket/2409)  but here's a workaround:

class SerializableGetter(object):

    def __init__(self, name):

        self.name = name

    def __call__(self, target):

        return getattr(target, self.name)

class Parent(Base):

    # ...

    children = relationship('Child', 

                collection_class=lambda: MappedCollection(SerializableGetter('key')))

On Feb 16, 2012, at 10:54 AM, Alex Grönholm wrote:

So basically this comes down the operator.attrgetter not being serializable. I'm wondering why something like this would be serialized at all -- aren't collections part of the class definition?

--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/sgLu0zjWla0J.
To post to this group, send email to sqlal...@googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.

[Alex Grönholm]

Yup, this is exactly what I did just 2 minutes ago :)