Assuming something in the datacenter goes wacky, how long might it
take for an index to update? Tens of seconds, minutes, hours, days?
Thanks,
Jeff
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Robert
Say you associate a facebook id with an account. In M/S, you'd
probably have something like this:
class User {
@Id Long id;
long fbId;
...
}
...and then when a request arrives with a facebook id, you would query
for the user record. No user record? Create one. With eventual
consistency, this creates a larger window (with M/S it was small)
where you can get duplicate Users for the same fbId.
The solution to transactional integrity and strong consistency is to
add a FbId entity:
class FbId {
@Id String fbId;
long userId;
}
I've now got several of these mapping entities in place now. Using
global transactions to create the FbId and the User at the same time,
it seems to solve consistency issues entirely. I don't know how it
will perform yet under load, but obviously there's not heavy
contention in this situation so I would be surprised if the 2pc hurt
much.
I'm starting to notice several of these FbId-type mapping objects
showing up in my code as a way to force queries (for unique items)
into strong consistency. I'm guessing you could do this for
multi-item queries using a list property instead:
Instead of query(Thing.class).filter("color", someColor), you could
instead keep updating an entity like this:
class ColorThings {
@Id String color;
List<Key<Thing>> things;
}
...which feels upside-down but really has a lot of advantages. If you
put ColorThings in memcache, it's like a query cache which actually
updates properly.
Is anyone else noticing their code being pushed into this pattern by the HRD?
Jeff
The downside of this multiple-thing index entity is that (like a
parent-key) it limits throughput. And since there's a 2pc involved,
it probably limits throughput quite a lot...
Jeff
Jeff
It also allows you to avoid contention issues when the Things are
"frequently" updated, but the indexed values may not be.
Robert
Alfred, your next project idea: write some type of low-level
high-performance batcher providing crazy high write-rates to a single
entity group. Perhaps with that you (or we) could come up with a
higher performance way to maintain global indexes. ;)
Robert
Oh, for those wondering what this thread is about... we're just making
up words / phrases.