error using memcache and filter()

[Subramanyam]

Hi

I am using memcached for caching objects , but am stuck with the
following

When I use the .all() method it works fine
>> from django.core.cache import cache
>>> user_list=User.objects.all()[0:10]
>>> key='userlist'
>>> cache.set(key,user_list)

But when I use .filter() method I get the following error

>>> user_list=User.objects.filter(is_staff=False)[0:10]
>>> key='userlist'
>>> cache.set(key,user_list)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/bvemu/lib/python2.6/site-packages/django/core/cache/
backends/memcached.py", line 37, in set
self._cache.set(smart_str(key), value, timeout or
self.default_timeout)
File "/home/bvemu/lib/python2.6/site-packages/python_memcached-1.45-
py2.6.egg/memcache.py", line 515, in set
return self._set("set", key, val, time, min_compress_len)
File "/home/bvemu/lib/python2.6/site-packages/python_memcached-1.45-
py2.6.egg/memcache.py", line 725, in _set
store_info = self._val_to_store_info(val, min_compress_len)
File "/home/bvemu/lib/python2.6/site-packages/python_memcached-1.45-
py2.6.egg/memcache.py", line 697, in _val_to_store_info
pickler.dump(val)
PicklingError: Can't pickle <class
'django.utils.functional.__proxy__'>: attribute lookup
django.utils.functional.__proxy__ failed

please let me know if anyone has seen the error before

Thanks
Subramanyam

[Frank DiRocco]

'yam,

I have not seen this error, but my responses to the error are inline.

On Jan 6, 2010, at 6:30 AM, Subramanyam wrote:

> File "/home/bvemu/lib/python2.6/site-packages/django/core/cache/
> backends/memcached.py", line 37, in set
> self._cache.set(smart_str(key), value, timeout or
> self.default_timeout)

So it appears your issue originates when you call cache.set
(key,user_list)

> File "/home/bvemu/lib/python2.6/site-packages/python_memcached-1.45-
> py2.6.egg/memcache.py", line 515, in set
> return self._set("set", key, val, time, min_compress_len)
> File "/home/bvemu/lib/python2.6/site-packages/python_memcached-1.45-
> py2.6.egg/memcache.py", line 725, in _set
> store_info = self._val_to_store_info(val, min_compress_len)
> File "/home/bvemu/lib/python2.6/site-packages/python_memcached-1.45-
> py2.6.egg/memcache.py", line 697, in _val_to_store_info
> pickler.dump(val)
> PicklingError: Can't pickle <class
> 'django.utils.functional.__proxy__'>: attribute lookup
> django.utils.functional.__proxy__ failed

But from these last three lines you can deduce that "Can't pickle" is
telling you the object is not unserializable (only in the failing
case). So, when we reflect back on the point at which you populated
the val you passed to the cache backend you will probably find:

>>> user_list=User.objects.all()[0:10]

produces a list of objects that are serializable (ie, your superuser
account)

and

>>> user_list=User.objects.filter(is_staff=False)[0:10]

is returning an unserializable value possibly? (although pickle.dump
([], some_object) succeeds, serializing an empty list)

I would have to say it seems like you are passing unexpected data to
the cacher that is not being validated as serializable and the pickler
is failing to serialize the invalid data. But this is, just a guess...

[Subramanyam]

Thanks a LOT !! Frank

Further to your analysis I checked the following http://docs.djangoproject.com/en/dev/topics/serialization/#serializing-data and understood from the note (rather my understanding ) that only .all() query set method  would give the serialized data

if we want .filter () ( or any other queryset methods didn`t check them all )  we have to serialize/de-serialize the queryset

Did the following and it worked

from django.core import serializers
cache.set('test1',serializers.serialize("json",User.objects.filter(is_staff=False)[0:10]))

there on I deserialize again to get the objects back


Thanks again !!


Regards
Subramanyam

is returning an unserializable value possibly? (although pickle.dump([], some_object) succeeds, serializing an empty list)

I would have to say it seems like you are passing unexpected data to the cacher that is not being validated as serializable and the pickler is failing to serialize the invalid data. But this is, just a guess...

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.