[web2py] PicklingError: Can't pickle <class 'Foo'>: it's not the same object as Foo

[MikeEllis]

The class isn't really named Foo, of course, but it makes for a more
readable subject line.

Here's the actual ticket:

Traceback (most recent call last):
File "/Users/mellis/web2py/gluon/main.py", line 504, in wsgibase
session._try_store_on_disk(request, response)
File "/Users/mellis/web2py/gluon/globals.py", line 375, in
_try_store_on_disk
cPickle.dump(dict(self), response.session_file)
PicklingError: Can't pickle <class
'applications.peertool.modules.PTProblemClass.Problem'>: it's not the
same object as applications.peertool.modules.PTProblemClass.Problem

And here is a simple default.py that reliably reproduces the error:

import cPickle
PTProblemClass = local_import('PTProblemClass',reload=True)

def mytest():

## Create an instance of class Problem and prove that it
## can be pickled.
problem = PTProblemClass.Problem("Just testing ...",ownerid=2)
filename = 'problem_' + str(problem.uuid)
PROBLEMFILEPATH =
os.path.join('applications','peertool','private')
f = open(os.path.join(PROBLEMFILEPATH,filename),'w')
cPickle.dump(problem,f)
f.close()

## Assign to session.problem and show that it is
## an instance of the class
session.problem = problem
assert isinstance(session.problem, PTProblemClass.Problem)

## Return a message of comfort and cheer
return dict(message=T("Test completed"))


def index(): ## Unaltered from new app creation
"""
example action using the internationalization operator T and flash
rendered by views/default/index.html or views/generic.html
"""
response.flash = T('Welcome to web2py')
return dict(message=T('Hello World'))

I've not included the Problem class code. It's proprietary and I'm
reasonably convinced its not part of the problem since there are no
problems pickling (or unpickling) instances of it.

I can reproduce the problem by the following steps:
1. Visit the index page, no ticket.
2. Visit mytest, no ticket
3. Visit index page again, get the ticket.

I'm running web2py 1.74.9.

Anyone know how to solve this? It's a particularly annoying because
once the error occurs, the entire app is hosed until I visit the site
page and clean the sessions and caches.

Thanks,
Mike

[MikeEllis]

More info:

I've confirmed that any class defined in the modules directory can
reproduce the problem. To verify, create a new module named
"dummyclass.py" containing

import uuid
class Problem(object):
""" Just to validate that session.problem pickling error
is not related to anything in PTProblem class.

"""
def __init__(self,description,ownerid=0):
self.uuid = uuid.uuid1()


and alter the local_import statement in my previous post to look like:

PTProblemClass = local_import('dummyclass',reload=True)

Thanks,
Mike

[MikeEllis]

Oops forgot to include "import os" in my example code. It was imported
elsewhere in my app. With that correction, I have now reproduced the
problem in a new app containing nothing but the example code.

I've tried googling the PicklingError message and found some
references to problems under WSGI, but they seemed to be GAE related
and the explanations weren't very clear (or at least not easy to
understand :-)

Any help much appreciated,
Mike

[mdipierro]

in web2py you CANNOT store an object into a session unless you store
it already serialized. This is because the session is retrieved before
your code is executed and therefore before the module in question is
imported.

[MikeEllis]

Thanks, Massimo. Makes perfect sense now that you've explained it.
The nature of my app is such that I really do need to keep the object
in the session, so just to make sure I understand, I need to do
something like

session.problem = cPickle.dumps(problem)

before leaving any function that alters the object, and

problem = cPickle.loads(session.problem)

on entry to any function that needs to use it?


Cheers,
Mike

[mdipierro]

yes.

[MikeEllis]

It seems like the best approach, for my app at least, is to define a
decorator to handle the serializing and unserializing of objects
stored in session, e.g.

@sessionobjects
def mycontroller():
...

This seems to be working, but I'm wondering if it has any known
gotchas relative to the under-the-hood workings of web2py. For
example, will the session always remain unserialized until after the
view is rendered? Or within the context of a redirect?

If this implementation seems sound, I'll post it in a comment to the
sessions page of the online doc.

Here's the code for the decorator.

## -----------------------------------------------------
def sessionobjects(f):
"""
Web2py sessions can't store app-defined objects unless they are
pre-serialized.
This decorator ensures that a list of session member objects are
unserialized before use and re-serialized afterward.
It also serves as a default initializer for the members, setting
them to None if they don't exist.
"""

def new_f():
objnames = ["problem",] ## edit list to fit your app
for n in objnames:
## In normal python, you would use hasattr() to see
## if the member exists, but session is of class Storage
## which returns None if the member doesn't exist.
if None != getattr(session,n):
setattr(session,n,cPickle.loads(getattr(session,n)))

try:
return f()
finally:
for n in objnames:
setattr(session,n,cPickle.dumps(getattr(session,n)))


return new_f

## -----------------------------------------------------