What does the new --high_replication flag actually do?

[Gwyn Howell]

In version Version 1.5.1 just released, one of the updates was:

The development server's datastore implementation now contains logic that closely replicates the consistency guarantees of the High Replication datastore. To use, run the dev_appserver with the flag --high_replication set to True.

What does this actually do? There must be an advantage of turning it on, but I can't see any difference. Is it just a different way to store the data in the local datastore?

[Robert Kluin]

Hi Gwyn,
It basically adds a small delay before a transaction applies, but
with facilities to grab consistent snapshots of entity groups when
needed. If you're not seeing any difference, then you're app is
probably doing ancestor queries or gets by key -- which is good.

You can check out the code:
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/datastore/datastore_stub_util.py#1310

Here is some code to demonstrate what the option does:

import uuid
import logging

class T(db.Model):
index = db.StringProperty()

for _ in xrange(100):
index = str(uuid.uuid4())
T(key_name=index, index=index).put()

# put a time.sleep(xx) here to see how it works.

# This will usually not work on prod HR datastore.
t = T.all().filter('index', index).get()
if not t:
logging.info('NoTing for you.')

# This should always work.
t_by_key = T.get_by_key_name(index)
if not t_by_key:
logging.error('You should never see this')

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/AAOJsS2h-FIJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

[Alfred Fuller]

Thanks Robert!

This feature is not an optimization, it is for testing. It allows you to see ruffly how your application will perform when deployed to an app that is using the High Replication Datastore. It works by estimating the likelihood of a write showing up in a global query after some amount of time. For example, if you ran the old guest book example code (which does not use entity groups) you are very unlikely to see a post you just made when redirected back to the guestbook page (I believe the example code has been updated to use entity groups). There are still some differences between the dev server and production:

• In the dev server a global query after a get or ancestor query will always see the current value (not so in production)
• In the dev server a global query will never return a result that is more than one write behind (not so in production)
• In the dev server you will never see an old value once a new value is seen (not so in production)

 - Alfred