What to do I must persist two different databases during commit / external-decider

[Hanns Holger Rutz]

Hi,

I need some advise on database persistence and external decider. I have
a scenario now, where I want to copy objects from one system to another,
each being a custom wrapper around a ScalaSTM transaction coupled to a
database transaction (BerkeleyDB JE).

The problem is that Berkeley doesn't allow transactions to span more
than one database environment. So far this wasn't a problem, and I set
the external decider to the `commit` function of the database.

Now with the copying scenario, I really need to open two database
transactions, but if one of them fails during `commit`, I might have
already committed the other one without any possibility of roll-back.

The mechanism below is the best I could come up with. Because the
inconsistency will be between the already committed database and the
in-memory ScalaSTM cells that got rolled back because of the second
transaction, perhaps the only clean solution is to mark the system as
corrupt and refuse any further operation, until the database is
re-opened and in-memory representations refreshed, so consistency is
restored.

Any thoughts?

Best, .h.h.



object TxnHelper {
private[this] final class Decider(
var instances: List[ExternalDecider])
extends ExternalDecider {

def shouldCommit(implicit txn: InTxnEnd): Boolean =
instances.forall(_.shouldCommit)
}

private val metaDecider = TxnLocal[Decider]()

def addExternalDecider(decider: ExternalDecider)
(implicit tx: InTxn): Unit =
if (metaDecider.isInitialized) {
metaDecider().instances ::= decider
} else {
val meta = new Decider(decider :: Nil)
ScalaTxn.setExternalDecider(meta)
metaDecider() = meta
}
}

[Hanns Holger Rutz]

So this would be my "improved" version which closes the databases which
will possibly run out-of-sync:

object TxnHelper {
trait Resource extends Closeable with ExternalDecider

private class Decider(var instances: List[Resource])

extends ExternalDecider {

def shouldCommit(implicit txn: InTxnEnd): Boolean =

instances match {
case single :: Nil => single.shouldCommit
case _ => commitAll(instances, Nil)
}

@tailrec
private def commitAll(remain: List[Resource], done: List[Resource])
(implicit txn: InTxnEnd): Boolean =
remain match {
case head :: tail =>
if (head.shouldCommit) {
commitAll(tail, head :: done)
} else {
if (done.nonEmpty) {
Console.err.println(s"Resource $head failed to commit"
+ " transaction.")
done.foreach { r =>
Console.err.println(s"Closing $r as a precaution."
+ " The system must be re-opened prior to further"
+ " use.")
try {
r.close()
} catch {
case NonFatal(e) => e.printStackTrace()
}
}
}
false
}

case _ => true
}
}

private val decider = TxnLocal[Decider]()

def addResource(resource: Resource)(implicit tx: InTxn): Unit =
if (decider.isInitialized) {
decider().instances ::= resource
} else {
val d = new Decider(resource :: Nil)
ScalaTxn.setExternalDecider(d)
decider() = d