prefetch_refprops doesn't work on Production, locally works as expected

[Chris Davis]

Hi,

I'm using the code below from Nicks' blog (http://blog.notdot.net/
2010/01/ReferenceProperty-prefetching-in-App-Engine).

Everything works as expected locally, I can see one request in /_ah/
stats for getting 100 items. However when I run the same code on
Production, I can see this method getting called 100 times in /_ah/
stats and can see my database read allocation decrease very quickly.

Am I missing something with the setup, or is there a difference in
behaviour between local and production?

Thanks

Chris

def prefetch_refprops(entities, *props):
fields = [(entity, prop) for entity in entities for prop in props]
ref_keys = [prop.get_value_for_datastore(x) for x, prop in fields]
ref_entities = dict((x.key(), x) for x in db.get(set(ref_keys)))
for (entity, prop), ref_key in zip(fields, ref_keys):
prop.__set__(entity, ref_entities[ref_key])
return entities

[Chris Davis]

This can be demonstrated with this trivial example, on Production 6
RPC's are generated, on Local, only 2

class Person(db.Model):
firstname = db.StringProperty(required=True)

class Names(db.Model):
userKeys = db.ListProperty(db.Key)

a = Person(firstname="adam")
b = Person(firstname="bob")
c = Person(firstname="charles")
d = Person(firstname="derick")
e = Person(firstname="edward")

people = [a,b,c,d,e]

db.put(people)

peopleKeys = [a.key(),b.key(),c.key(),d.key(),e.key()]

n = Names()
n.userKeys = peopleKeys

items = db.get(n.userKeys)

[--drop table]

Hi, Has anyone been able to confirm this? or understand why this
happens?

Thanks

Chris

[Bryce Cutt]

As Nick said on his blog:

http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine#comment-533928983 

"""
The SDK was modified a while ago to send one request per entity group in production. The requests are sent asynchronously, and the new method is actually faster than sending them all in one batch request.

The quota usage is the same regardless - you use one read operation per entity you read.