Experience about storing a dict in an entity

[manatlan]

I've needed to store a dict (1000 key/values) in an entity (value is a
tuple of unicode, float and datetime. key is a string)

My first try was to pickle it and to store it in db.Blob. Really
simple, but pickle is slow, and consumed a lot of CPU cycle
so I decided to avoid pickle, and build my own serializer/
deserializer ....
My second try, was to transform my dict as a list (with a homemade
serializer), and store it in a
db.ListProperty( db.StringProperty ) ... But it was a nightmare in
consuming CPU cycle. All was consumed by google api with the
listproperty of string. First solution was a lot better.
My last try, was to transform my dict as a string (with a homemade
serializer), and to store it in a db.TextProperty ... the better way
(and from far !)

The nightmare was really during the second solution. "ListProperty of
Stringproperty" consume a lot of cpu cycle ... (with 1000 strings) ...

If 1st solution as a ration of 1. Second was 1.5 ... and third/last is
0.5.

And the big nightmare is that I've done my tests in production, on the
default version (sic ;-)... with people on website ... during the
second test : a lot of 403 quotas gives me a lot headaches ;-)....

[Calvin Spealman]

I can think of two solutions.

1) Build your own persistent mapping type that defines __*item__
methods and is a subclass of db.Expando. Store them as dynamic
attributes with some special prefix. This works because your keys are
all strings.

2) Build such a type, but as a db.Model and an entry type, as well.
Entries reference their dictionary and know their key and value. Index
on the key of course.

--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

[Drew Sears]

For my purposes, I serialize/deserialize to strings with
django.utils.simplejson. I doubt this would be more performant than
the stuff you've already tried, though.