Consistently getting inconsistent reads in GAE Java DataStore

[Rémi Bastide]

Several people have detected a strange behaviour in Google App Engine JDO, where your app unpredictably returns stale data.

I've written a small servlet to demonstrate this.

The code is included in this note, and the app uses the plain vanilla jdoconfig.xml also included here.

You can see it live deployed in appengine here 

When using this app, you will notice that occasionally the records that have just been added or deleted do not show up until a couple of refreshes.

Things that have been tried but do not appear to work : 

• Wrapping transactions around adds and deletes
• Using cache control headers in response
• setting <property name="datanucleus.appengine.datastoreReadConsistency" value="STRONG" /> in jdoconfig.xml

I suspect a threading issue that can be solved by configuration options... 

I must be doing something terribly wrong, but I do not see what...

Any help appreciated

[Joshua Smith]

If you are using HRD, this is a well-documented limitation.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/3nebSrKk9G4J.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
<StaledataServlet.java><jdoconfig.xml>

[Simon Knott]

As Joshua mentions, if you are on HRD you are simply experiencing eventual consistency.  Unless you do gets by keys, which will always give you up-to-date data, you will potentially experience stale data due to the time it takes to replicate the entities and indexes across the GAE replicated datastore.

Cheers,

Simon

[André Pankraz]

Even with "gets by key"...may be I'm wrong, but you can put/update an entity, commit/end the transaction and a following "get entity by key" may return stale data - right?
Entity replication takes it's time too - index updates are delayed even further (link explains it, 2 steps) and you encounter the effect more often.

Or not? I find the documentation to this topic annoying and confusing - and for me it's the major pain point with the data store. Complicates write logic etc.

[Simon Knott]

Gets by key always return consistent data, unless you specifically force it to be eventually consistent.  Similarly, ancestor queries on the HR datastore always give you the correct data.  Both are wrapped in transactions in the background, I believe.

The overview on http://code.google.com/appengine/docs/python/datastore/hr/ describes which elements have strong and weak consistency on the different datastores.

[Rémi Bastide]

Thanks to you all for the input. However, I'm quite surprised :

• I disabled "Local HRD support" in "Appengine properties" (I'm using Eclipse plugin) 
  
• The app now appears to work normally (i.e showing no stale data) when run locally
• My appengine app bastide07 indicates " Master/Slave Replication"
• The problem of stale data still persists when run on appengine, see : http://bastide07.appspot.com/staledata

I