Unexpected Entity Group transaction contention

[Brian Olson]

Re-reading the documentation, this kinda makes sense, but it bit me recently so I want to tell the story and see what others think.

I make an entity Parent(). Some time later I make an entity Child(parent=some_parent) and I do this in a transaction. I do this a bunch, concurrently from task-queue entries.

I was surprised to learn that simply creating a Child in a transaction, without otherwise doing anything to the parent, neither .get() nor .put(), locks the parent and all its children.

def txn_make_child(some_parent):

  foo = Child(parent=some_parent)

  foo.put()

  # also transactionally enqueue a task to operate on the Child instance foo

Code very much like that was failing out due to too many transaction retries. I didn't expect any transaction contention, because I thought I was just creating an object and enqueueing a task, and those were the only two things in the transaction in my head. But it turns out the above code locks some_parent and all its children. Boo.

I think I was expecting things like this to lock parent and all its children:

def txn_p_c_example(parent_key, child_key):

  parent = db.get(parent_key)

  child = db.get(child_key)

  # now they're clearly both involved, and involving the parent winds up locking all the children. I can accept that.

  parent.put()

  child.put()

I was able to re-code it to make Child have no ancestor, but there are still times when I would much rather still commit parent and child at exactly the same time.

[Steve Sherrie]

Testing is happening presently for multi-entity-group transactions I believe. I saw a thread on it about a week ago.

Steve

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/NsvS8Fcq_EwJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Mike Wesner]

Any time you write/update an entity in a group (share a common
ancestor) you lock the entire entity group all the way to the top most
parent. All branches under that parent... the entire tree.

Multi eg transactions will allow you to write to multiple groups but
it will not reduce contention in this scenario at all.

Mike

On Sep 13, 2:23 pm, Brian Olson <br...@cloudlock.com> wrote:
> Re-reading the documentation, this kinda makes sense, but it bit me recently
> so I want to tell the story and see what others think.
>
> I make an entity Parent(). Some time later I make an entity
> Child(parent=some_parent) and I do this in a transaction. I do this a bunch,
> concurrently from task-queue entries.
>
> I was surprised to learn that simply creating a Child in a transaction,
> without otherwise doing anything to the parent, neither .get() nor .put(),
> locks the parent and all its children.
>
> def txn_make_child(some_parent):
>   foo = Child(parent=some_parent)
>   foo.put()
>   # also transactionally enqueue a task to operate on the Child instance foo
>
> Code very much like that was failing out due to too many transaction

> retries. I didn't expect *any* transaction contention, because I thought I

[Brian Olson]

Multi-entity transactions would allow me to redesign my app to not have the parent-child relationship and then the contentious operations on children would be just fine as those child entities would be unrelated in the new version.

I'm sure multi-entity transactions are going to come with tradeoffs like "more likely to retry/fail" and "slower".