About expando properties

[Pol]

Hi,

Regarding expando properties:

1) Can you prevent them from being indexed (I couldn't find anything
about this, but I'd like official confirmation)?

2) What's the practical number of expando properties you shouldn't go
above? If they are all indexed, then surely that can be quite
expensive, so you should stay below 100 or something?

3) If you have to be above the limit defined at #2, what's the best
alternative? Serializing into a blob property, possibly, but what
would be the most optimal serialization format?

Thanks!

[Max]

1) text property? not familer with Python, I believe there is a better answer

2) max 5000 indexes are allowed to build on single entity

3) you can try to save all properties (those don't require index) into one single json / xml text property

[Prateek Malhotra]

I just looked into this myself. Best bet is to use a Text or Blob
property to store in a JSON format:

http://kovshenin.com/archives/pickle-vs-json-which-is-faster/

I did many tests with 250-500 expando properties and the results were
horrendous. From digging around I didn't find a way to disable
indexing expando properties, and this made writing them very
expensive.

[Jay Young]

Instead of setting an expando property to the property's value, can you set it to an instance of a Property class?  To modify the example in the documentation:

class Song(db.Expando):
    title = db.StringProperty()

crazy = Song(title='Crazy like a diamond',
             author=db.StringProperty('Lucy Sky', indexed=False),
             publish_date='yesterday',
             rating=5.0)

[Pol]

Very interesting, thanks for sharing.

If you just need a simple dictionary-like storage, why not just repr()
and eval() - wouldn't that be even faster?

[keakon lolicon]

There is an _unindexed_properties private property of model class:

class A(db.Expando):

  pass

A._unindexed_properties = frozenset(['prop'])

a = A()

a.prop = 1 # it's not indexed

a.put()

----------
keakon

My blog(Chinese): www.keakon.net

Blog source code: https://bitbucket.org/keakon/doodle/

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Prateek Malhotra]

While this seems to be the case in Python 2.5 (without a built-in json
class), it isn't true in Python 2.7 (from my tests at least):
############
import string, random, profile, json
#import simplejson as json

def randomString(length):
return ''.join(random.choice(string.ascii_uppercase + string.digits)
for x in range(length))

t = {}
for x in range(500):
t[randomString(8)] = randomString(12)

profile.run("eval(repr(t))")
profile.run("json.loads(json.dumps(t))")
############

Using the simplejson class resulted in code significantly slower
(15-20x slower). However, using the built-in json class in Python 2.7
resulted in code slightly faster than repr() (3-5x faster).

With the upcoming support for 2.7, I'd suggest keeping json encoding.
It's also more portable "as-is" although it wouldn't be such a hard
task to switch between the two.

[Geoffrey Spear]

On Jul 14, 9:36 am, someone1 <someo...@gmail.com> wrote:
> Using the simplejson class resulted in code significantly slower
> (15-20x slower). However, using the built-in json class in Python 2.7
> resulted in code slightly faster than repr() (3-5x faster).
>
> With the upcoming support for 2.7, I'd suggest keeping json encoding.
> It's also more portable "as-is" although it wouldn't be such a hard
> task to switch between the two.

Keep in mind that Python 2.7's json module might be using C speedups
that might not be available on App Engine, and it's actually the same
codebase as simplejson (except that the version in Python 2.7 lags
behind simplejson a bit).