Concurrency: hypothetical variables and atomic blocks
Jonathan Lang
implicitly hypotheticalized? I'm thinking that a "retry" exit
statement may be redundant; instead, why not just go with the existing
mechanisms for successful vs. failed block termination, with the minor
modification that when an atomic block fails, the state rolls back?
Also, what can "retry_with" do that testing the atomic block for
failure can't?
--
Jonathan "Dataweaver" Lang
Daniel Hulme
> implicitly hypotheticalized?
entry, it also applies to I/O and other side-effecty things that you
can't undo.
--
Hats are no worse for being made by ancient hatters, and good butter can
be bought in a shop that has no jazz-band. -- J.B. Morton (Beachcomber)
P.S. --- Tell Saunders that he must not leave cat's meat on my desk.
http://surreal.istic.org/ old-fashioned quality within the McQuary limit
Sam Vilain
>How does an atomic block differ from one in which all variables are
>implicitly hypotheticalized? I'm thinking that a "retry" exit
>statement may be redundant; instead, why not just go with the existing
>mechanisms for successful vs. failed block termination, with the minor
>modification that when an atomic block fails, the state rolls back?
>
>
State rolling back automatically is the key feature of STM.
However, it can only cover pure perl state; any time that you enter a
function that performs I/O of any kind, then you are forcing bad things
to happen.
With Haskell this is sorted out by making the default "pure", and
everything else must be in a Monad. However we're not into bondage here
so "is pure" is not default. Instead we just die and rollback just
before I/O is attempted. In principle, the compiler could automatically
attach "pure" traits to all functions and methods that it can prove will
never perform I/O and warn about this at compile time.
It might be possible for clever classes to circumvent this and carefully
call special unsafeIO() methods, and be passed messages about the STM
and hope that they do the right thing. I don't know that anyone's
explored this area in depth in Perl 6 space.
>Also, what can "retry_with" do that testing the atomic block for
>failure can't?
>
I think the answer lies in the "checkpointing" references in that
document. I don't know whether that's akin to a SQL savepoint (ie, a
point mid-transaction that can be rolled back to, without committing the
entire transaction) or more like a continuation that when resumed can
see the atomic changes, and when exiting finally applies them (or rolls
back). Perhaps someone else will have more of a clue.
Sam.
Sam Vilain
>>How does an atomic block differ from one in which all variables are
>>implicitly hypotheticalized?
>>
>>
>I assume that the atomicness being controlled by some kind of lock on
>entry, it also applies to I/O and other side-effecty things that you
>can't undo.
>
The lock on entry approach will only be for non-threaded interpreters
that don't know how to do real STM.
Sam.
Darren Duncan
>I think the answer lies in the "checkpointing" references in that
>document. I don't know whether that's akin to a SQL savepoint (ie, a
>point mid-transaction that can be rolled back to, without committing the
>entire transaction) or more like a continuation that when resumed can
>see the atomic changes, and when exiting finally applies them (or rolls
>back). Perhaps someone else will have more of a clue.
>Sam.
Rather than thinking about "save points", it would be better to think
of the problem in terms of "child transactions". (Note that what I'm
saying here is simplified to assume there aren't any irreversable
actions, but it can easily be extended to handle those situations
too.)
Each time a context (a code block, either a routine or a syntactic
construct like 'try' is) is entered that is marked 'is atomic', a new
transaction begins, which as a whole can later be committed or rolled
back; it implicitly commits if that context is exited normally, and
it rollsback implicitly if the context exits with a 'fail' and/or due
to a thrown exception. (And yes, I see it as being easier to use if
rollback and fail are generally joined at the hip.)
One atomic context can contain another atomic context, so they are
layered; if an outer layer rolls back, it results in any 'successful'
inner layers also rolling back, so that everything which happened
within the outer context has rolled back.
If we simply have child atomic contexts to implement sub-transactions
of a parent atomic context / transaction, rather than relying on
savepoints or whatever, then it is much easier to make reusable or
movable or recursive code, since the code doesn't have to specify its
own atomicness differently depending on whether its caller is being
atomic or not.
Eg, we could have a situation like this:
sub foo is atomic { ... }
sub bar is atomic { ... }
sub baz is atomic { ... }
sub quux is atomic {
...
foo();
...
try {
bar();
}
catch {
baz();
}
...
}
quux();
All 4 of the above subroutines are individually atomic and will throw
an exception / return 'fail' and rollback on failure. If quux()
fails or foo() or baz() fail, nothing that either of those 3
subroutines did will persist. If bar() fails, but baz() succeeds,
then only what bar() did has rolled back, and the rest persists when
quux() returns.
Those are my thoughts concerning transactions. I haven't
specifically addressed any matters related to exclusivity of
resources in the cases of multiple processes or threads, but they can
easily be added onto what I stated, which is also useful when there
is just a single process with a single thread.
-- Darren Duncan
Ruud H.G. van Tol
> Each time a context (a code block, either a routine or a syntactic
> construct like 'try' is) is entered that is marked 'is atomic', a new
> transaction begins, which as a whole can later be committed or rolled
> back; it implicitly commits if that context is exited normally, and
> it rollsback implicitly if the context exits with a 'fail' and/or due
> to a thrown exception. (And yes, I see it as being easier to use if
> rollback and fail are generally joined at the hip.)
There are also 'lazy atomic' actions that may not achieve (yet) what
they are supposed to do (maybe the storage of some memory structure to a
disk file). So they sort of 'try' and then sort of 'fail', but they
shouldn't block continuation yet, because they will get another go at it
later on.
> One atomic context can contain another atomic context, so they are
> layered; if an outer layer rolls back, it results in any 'successful'
> inner layers also rolling back, so that everything which happened
> within the outer context has rolled back.
If the 'results' would be (for example) text data written to a logfile,
the 'roll back' could be (1) removal of the written data, or (2)
appending a line about the failure. But writing that last line can of
course also fail.
--
Groet, Ruud
Larry Wall
: The lock on entry approach will only be for non-threaded interpreters
: that don't know how to do real STM.
The way I see it, the fundamental difference is that with ordinary
locking, you're locking in real time, whereas with STM you potentially
have the ability to virtualize time to see if there's a way to order
the locks in virtual time such that they still make sense. Then you
just pretend that things happened in that order.
As for side-effecty ops, many of them can just be a promise to perform
the op later when the transaction is committed, I suspect.
Larry
Jonathan Lang
> The way I see it, the fundamental difference is that with ordinary
> locking, you're locking in real time, whereas with STM you potentially
> have the ability to virtualize time to see if there's a way to order
> the locks in virtual time such that they still make sense. Then you
> just pretend that things happened in that order.
Forgive this ignorant soul; but what is "STM"?
--
Jonathan "Dataweaver" Lang
Jonathan Scott Duff
Jonathan Worthington
an interesting paper here:-
http://www.cambridge.intel-research.net/~rennals/faststm.html
Jonathan
Darren Duncan
>As for side-effecty ops, many of them can just be a promise to perform
>the op later when the transaction is committed, I suspect.
Yes, but it would be important to specify that by the time control is
returned to whatever invoked the op, that any side effects will have
been successful as well, or a failure/exception is still thrown.
What happens in Perl itself and what happens as external side effects
are tied together from the invoker's point of view as one unit that
should entirely succeed or entirely fail. Even if the side-effects
are put off as late as possible in the transaction, we still need to
know whether they succeeded or not.
On a related note, if there is some external system used in a
transaction that can't guarantee a successful rollback on failure,
then any error returned by the Perl op to its invoker should
differentiate between whether a failure included a successful
rollback in the external system, or whether said system is now
possibly or actually in an inconsistent state, so the invoker knows
whether or not it should be safe to proceed. Similarly, if the
external system can't guarantee successful completion before it
returns control, the invoker should know that vs when completion is
guaranteed.
In other words, an invoker of an op should know whether the op is
ACID compliant in all parts of its operation or not.
-- Darren Duncan