Java STM

[Vagif Verdi]

Potentially interesting library for clojurians. Java STM
implementation: http://www.deucestm.org/

[Mark Volkmann]

On Sun, Jul 12, 2009 at 7:07 PM, Vagif Verdi<Vagif...@gmail.com> wrote:
>
> Potentially interesting library for clojurians. Java STM
> implementation: http://www.deucestm.org/

As best I can tell, this is yet another "on your honor" STM
implementation. What I mean is that as long as you use the library
correctly in every place where it is needed, everything works fine.
Forgot to use it somewhere in an application where it was needed and
all bets are off. That's one thing I really like about Clojure. You
can't get away with forgetting to update Refs inside a transaction. If
you do, you'll be reminded with an exception.

Is there another STM implementation that enforces its use like this?

--
R. Mark Volkmann
Object Computing, Inc.

[Jon Harrop]

On Monday 13 July 2009 01:55:07 Mark Volkmann wrote:
> Is there another STM implementation that enforces its use like this?

I assume Haskell tells you at compile time.

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

[Christian Vest Hansen]

I believe that DeuceSTM i primarily intended as a research platform
for Java STMs, hence the flexibility with pluggable algorithms.

Another Java STM is multiverse: http://code.google.com/p/multiverse/ -
the focus here is on performance. Multiverse is based on MVCC, like
the Clojure STM.

Both of these STMs operates with the mindset that objects are mutable
by default, and tries to control this mutability. Whereas in the world
of Clojure we have mutable references pointing to immutable values. In
fact, the correctness of the Clojure STM depends on the objects
"stored" in the refs be immutable.

On Mon, Jul 13, 2009 at 2:07 AM, Vagif Verdi<Vagif...@gmail.com> wrote:
>
> Potentially interesting library for clojurians. Java STM
> implementation: http://www.deucestm.org/
>
> >
>

--
Venlig hilsen / Kind regards,
Christian Vest Hansen.

[Daniel]

Well, I'd say that's a property of the language, not the library. You
could use pretty much any STM library in Java if you can make sure
that the executed bytecode has the appropriate calls in place. Have a
look at Terracotta (http://terracotta.org/). It's not an STM in the
sense of intra-JVM transactions, but inter-JVM transactions (they call
it network attached memory), and it's implemented as a bytecode weaver
(meaning you either transform your compiled class files with an
additional step after compilation to add the library specific
instructions, or you do the same thing at loadtime when loading your
code through a special classloader). Terracotta's approach is also "on
your honor" in the sense that it doesn't force you to declare all
variables in TC terms, but the approach could be extended to throw
(compile time) exceptions on undeclared state variables.

By the way, I would take a good look at Terracotta if I ever wanted to
extend Clojures STM to work across JVMs. You probably would have to
think a bit harder about how you integrate the TC transactions with
the ones from Clojure, since TC transactions are remote calls and
therefore a lot slower, but I reckon it shouldn't be too hard. TC has
enough tools to inspect how your transactions perform.

HTH

Cheers,
Daniel

[Daniel]

Oh, and just to complete the picture for intra-JVM transactions on
shared state (clustering), besides Terracotta there's also Hazelcast
(http://www.hazelcast.com/), and to a lesser degree (different focus)
ZooKeeper (http://hadoop.apache.org/zookeeper/) and Shoal
(https://shoal.dev.java.net/). But you'll have to check what the
libraries are aimed at, since the implementations are designed for
certain usecases. And all of these are implemented and used as
traditional libraries as opposed to Terracotta.

[peter veentjer]

Hi Christian,

On Jul 13, 10:37 am, Christian Vest Hansen <karmazi...@gmail.com>
wrote:

> I believe that DeuceSTM i primarily intended as a research platform
> for Java STMs, hence the flexibility with pluggable algorithms.
>
> Another Java STM is multiverse:http://code.google.com/p/multiverse/-
> the focus here is on performance. Multiverse is based on MVCC, like
> the Clojure STM.
>
> Both of these STMs operates with the mindset that objects are mutable
> by default, and tries to control this mutability. Whereas in the world
> of Clojure we have mutable references pointing to immutable values. In
> fact, the correctness of the Clojure STM depends on the objects
> "stored" in the refs be immutable.

Multiverse also provides a ref programming model.

So you could say this:

@AtomicObject
class Person{
private String name;

public String getName(){return name;}

public void setName(String newName){this.name = newName;}
}

Or using the ref programming model:

class Person{
private final Ref<String> name = new Ref<String>();

public String getName(){return name.get();}

public void setName(String newName){name.set(newName);}
}

If you have multiple fields and and a static language like Java,
the first approach programs better.

If a mutable reference is stores in the field the following things
could happen:
1) the object is not an atomicobject and you are left on your own. For
example a java.util.LinkedList stored in the ref.
The advantage of allowing 'unsafe' objects to be stored in the stm, is
that they can flow through the stm. Just like
a normal mutable object can flow through a
java.util.concurrent.BlockingQueue. But I'm certain that people are
going
to run into problems because they forget that the object is not
managed. Time will tell :)
2) the object is an atomic object and it will be managed by the stm as
well.

So something like this is no problem:

@AtomicObjec
class Person{Sttring name;...}

Ref<Person> parentRef = new Ref<Person>();

In this case the parent reference will be managed by the stm, but also
the content of the parent object itself.

A few other questions:

Does Clojure support the stm version of condition variables: retry and
orelse?

And since Clojure is using MVCC, does it also suffer from a broken
serialized isolation level?
http://pveentjer.wordpress.com/2008/10/04/breaking-oracle-serializable/

And what kind of performance do you get with clojure?

PS:
I think that Clojure really did a great job with adding STM to the
language. I have been working on multiverse for almost a year,
and I only have an stm. With Clojure the STM is one of the many
features provided.

[Rich Hickey]

No. I don't want to use transactions for workflow. I don't want
blocking transactions. I don't want read tracking.

> And since Clojure is using MVCC, does it also suffer from a broken
> serialized isolation level?http://pveentjer.wordpress.com/2008/10/04/breaking-oracle-serializable/
>

Clojure doesn't promise any "serialized isolation level". It provides
snapshot isolation, which is subject to write skew, as better
described here:

http://en.wikipedia.org/wiki/Snapshot_isolation

In order to prevent write skew, Clojure provides the 'ensure'
operation which can be used for reads to non-written refs that form
part of the integrity promise of a transaction, without incurring that
overhead all the time for reads that do not. I much prefer having this
fine-grained choice.

> And what kind of performance do you get with clojure?
>

I'll let other people answer that from their experience, but only say
that it is probably too general a question to be particularly
meaningful. Clojure's STM is part of a holistic language design where
people will normally be programming with immutable persistent
composite data structures. Getting a consistent view of such a data
structure doesn't require a transaction at all, so that is much faster
than other strategies. When 'changing' a data structure, you read it
from the ref once and then structural changes do not involve the STM
or refs. Store it back in the ref when done. Thus the granularity of
transactions is coarse and the number of ref interactions involved
small. These are not things you'll see when timing pounding an integer
in a transactional ref in a tight loop, but matter greatly in
practice, IMO.

Rich

[peter veentjer]

> No. I don't want to use transactions for workflow. I don't want
> blocking transactions. I don't want read tracking.

With multiverse it depends on the engine being used and the settings
on the transaction. And readonly transactions also don't track reads.

> > And since Clojure is using MVCC, does it also suffer from a broken
> > serialized isolation level?http://pveentjer.wordpress.com/2008/10/04/breaking-oracle-serializable/
>
> Clojure doesn't promise any "serialized isolation level". It provides
> snapshot isolation, which is subject to write skew, as better
> described here:
>
> http://en.wikipedia.org/wiki/Snapshot_isolation
>
> In order to prevent write skew, Clojure provides the 'ensure'
> operation which can be used for reads to non-written refs that form
> part of the integrity promise of a transaction, without incurring that
> overhead all the time for reads that do not. I much prefer having this
> fine-grained choice.

I'll have a look at it.

>
> > And what kind of performance do you get with clojure?
>
> I'll let other people answer that from their experience, but only say
> that it is probably too general a question to be particularly
> meaningful.

True.

> Clojure's STM is part of a holistic language design where
> people will normally be programming with immutable persistent
> composite data structures. Getting a consistent view of such a data
> structure doesn't require a transaction at all, so that is much faster
> than other strategies. When 'changing' a data structure, you read it
> from the ref once and then structural changes do not involve the STM
> or refs. Store it back in the ref when done. Thus the granularity of
> transactions is coarse and the number of ref interactions involved
> small. These are not things you'll see when timing pounding an integer
> in a transactional ref in a tight loop, but matter greatly in
> practice, IMO.

I partly agree. One of my design guidelines is that one should not
have to pay
for what is not being used.

One of the things I'm focussing on is making the transaction
as fast as possible for any length. I'm working on a system that
advices the
creation of a transaction with the maximum number of attached objects.
I have
a transaction optimised for a single attachment (5+M transactions/
second on a single core),
for a small number of attachments (so using an array to store attached
items to reduce object creation)
and one for a large number of attachments (so using an expensive
hashmap).
The later one also is going to get a parallel commit (so obtaining
locks/checking for
isolation problems.. and doing the write) to make effective use of the
cores and speed
up the commit of larger transactions.

So I'm working on all levels to make it as fast as possible and
pounding
on an intref is something that helps to find a subset of problem
areas. And if something
is fast enough, people don't try to work around it.. so it also helps
to reduce complexity imho.

But again, I think that clojure is doing a great job. You have a cool
language,
a cool stm, an active community and a lot of exposure. So you are
where I
want to be :)

[Emeka]

Peter, you will get there some day.

[Rich Hickey]

And I'm not arguing against fast STMs :) Just explaining that, because
my ideas about STM (you shouldn't need a transaction to see a
consistent object) required persistent data structures and a language
where they were pervasive and idiomatic, I ended up with a whole
language and an approach that includes certain efficiencies, which, I
think, will hold me (and Clojure users) until I (or someone) get
around to perf-tuning the STM, which is something I've spent no time
on at all. You know the old saws, "make it right, then make it fast",
"premature optimization is the root of all evil in programming"... I'm
still working on the "make it right" phase :)

Rich

[John Harrop]

On Fri, Aug 28, 2009 at 4:45 AM, peter veentjer <alarm...@gmail.com> wrote:

For a single "attachment" Clojure has atoms and agents instead of an optimized specialization of refs and dosync. :)