I update an entity, then reading with Request Factory I sometimes get the old value - High Replication Datastore (HRD)

[savilak]

I use JDO and I update an Entity like this:

pm.currentTransaction().begin();
Contact c = (Contact) pm.getObjectById(Contact.class,
encodedKey);
//JDOHelper.makeDirty(c, "groups");//Make a field dirty --
Comment 1
c.setGroups(groupsNew);
pm.makePersistent(c);
pm.currentTransaction().commit();
// Contact c1 = (Contact)
pm.getObjectById(Contact.class, encodedKey); -- Comment 2: This is
making the read consistent

then I use Request Factory to read this Entity. 50% of the times I get
the Old value and not the New one.

I know that is due to the High Replication Datastore (HRD). How can I
force the read always to be consistent?

One way I managed to do it is by adding an extra read of the entity
right after its update. I do not like this approach. Why exactly is
this happening?
I also noticed that the makeDirty approach does not resolve the issue
(see Comment 1).

I believe that forcing all the entities in the same entity group will
not be a good solution for my context either
(as it is described here
https://developers.google.com/appengine/docs/java/datastore/structuring_for_strong_consistency).
I have User Accounts one User Account might have up to 30.000
contacts. A typical query would be fetch a range of 100 contacts for a
specific User Account.
Entity Group will move parsing from Datastore to my application (since
I have no join facility in datastore). Is this correct?

I would much appreciate some input from an HRD guru or anyone who has/
d a similar issue.

Thanks
Savilak

[Michael Hermus]

As you pointed out, to guarantee strong consistency you need to use an Ancestor query, or retrieve an Entity by its Key value. In both cases, the data store should roll-forward any commits that have not yet been applied.

I am a little confused because it looks like you are retrieving the Entity by Key in your sample code, but I assume you are really saying that a (non-Ancestor) query down the line is returning inconsistent results. That is the expected behavior, and you have to design around it. The latency is usually a few seconds, although even that is not guaranteed.

While you are correct that 30,000 Entities in one group sounds like a lot, you may be able to get away with it if the Contacts are not updated frequently or concurrently. In other words, if only that single User can update his/her own contacts one at a time, there will never be any update contention within the Entity Group, which (AFAIK) is the main reason to limit Entity group size.

[savilak]

Michael, thank you for your comment.

Thus, the only way to make a strongly consistent update (no Ancestor
Entity scenario) is after the update to retrieve the entity by its key
value. This way all pending changes will be roll-forwarded, then read
again using Request Factory from client side (e.g. 100 contacts) will
find all the data on this specific entity committed.

If this is correct, there must be a more transparent way to instruct
Datastore (using JDO) to force roll-forward the commits for and entity
update (instead of reading the same entity by key again).

Note: Line // Contact c1 = (Contact) pm.getObjectById(Contact.class,

encodedKey); -- Comment 2: This is making the read consistent

IS COMMENTED OUT

On May 9, 9:39 pm, Michael Hermus <michael.her...@gmail.com> wrote:
> As you pointed out, to guarantee strong consistency you need to use an
> Ancestor query, or retrieve an Entity by its Key value. In both cases, the
> data store should roll-forward any commits that have not yet been applied.
>

> I am a little confused because it *looks *like you are retrieving the

> Entity by Key in your sample code, but I assume you are really saying that
> a (non-Ancestor) query down the line is returning inconsistent results.
> That is the expected behavior, and you have to design around it. The
> latency is usually a few seconds, although even that is not guaranteed.
>
> While you are correct that 30,000 Entities in one group sounds like a lot,

> you *may *be able to get away with it if the Contacts are not updated

> >https://developers.google.com/appengine/docs/java/datastore/structuri...).