Proposal: supporting GAE transactions in DAL

[Robin B]

To perform certain actions on GAE you need transactions, eg.
atomically increment a counter.

This is what it would look like:

row.update_record(count=lambda r: r.count+1)

The GAE driver would check attributes for lambdas and invoke a
transaction when a lambda is present, the other SQL drivers could be
modified to be tolerant of lambdas and pass the current row into the
lambda and will get the correct outcome.

The transaction would work something like this in gql.py:

from google.appengine.ext import db as google_db

def update_record_txn(attrs,table_obj):
record = table_obj.get_by_id(attrs.id)
for k,v in attrs:
if isinstance(v,lambda x: x):
v = v(record)
setattr(record,k,v)
record.put()
return record

try:
return google_db.run_in_transaction(update_record_txn,
row,table_obj)
except:
pass
# could handle TransactionFailedError() caused by contention

This would be backwards compatible.

Thoughts?

Robin

[mdipierro]

I think we have something like this without introducing a special
syntax for GAE based on lambda. In fact on non-gae you can now do

row.update_record(count=db.table.count+1)

The same could work on GAE with your code below.

massimo

[Robin B]

> row.update_record(count=db.table.count+1)

Nice, thats great syntax for incrementing counters, but you would need
a lambda to do anything more complicated than arithmetic in a
transaction.

What about a more complex transaction like read a value and set it
based on rules (for creating a lock)? That could only be accomplished
with a lambda...right?

Robin

[mdipierro]

let me think about this...

[mdipierro]

The more I think about this the more I like your solution.

We can provide both notations:
1) row.update_record(count=db.table.count+1)
2) row.update_record(count=lambda r: r.count+1)
and the lambda notation should work even without GAE.

If you send me the patch for GAE I can fix the non-gae.

Massimo

[Robin B]

The next logical step is creating, reading, updating, and deleting
other records from within the transaction, at which point the lambda
which is really a function, should not be stuck under the field=...
since it is now much more than a field.

Since the lambda is passed the entire record, perhaps the syntax
should be more like:

def my_txn(record):
record = record.count+1
return record

row.update_record(my_txn)

the query checks if isinstance(obj,callable)...

Gotta run, Hasta Manana.

Robin

[Robin B]

What about:

def my_txn(db):
record = db(db.table.id==123).select()[0]
record.update_record(count=record.count+1)
return record.count

count = db.transaction(my_txn)

Robin

[mdipierro]

how would it be implemented if it does not know which record to lock?

[Robin B]

On GAE, when you start a transaction, it tries to lock the (1) entity
group for the record(s) that you access, and if you try to do
something illegal like query or access records from another entity
group, it will create an exception and the transaction will abort.

The transaction either succeeds, retries or aborts.

It would be best if the driver caught the possible exceptions and
returned success or failure.

def mytxn(db,id):
record = db(db.table.id==id).select()[0]
record.update_record(count=record.count+1)
return record.count


txn = db.create_transaction(my_txn)
if not txn(123):
raise Exception('transaction failed: %s'% txn.message)
count = txn.result



Robin