Provided value of using STM?
Peter Veentjer
I think that that distributed stm and a transparent storage engine are
going to be very important steps forwards. One of the goals is to make
application development a lot simpler, partially inspired by projects
I have done at in the past or helped customers with. I have done quite
a few 'integration' solutions that make it possible for systems to
exchange information. In these cases storage often something that is
needed, but the actual structure of the data is only interesting for
the application itself (so no need for an rdbms). Having a transparent
storage engine combined with an improved programming model for
concurrent and distributed systems should simplify these types of
applications dramatically.
My opinion is that most problems in enterprisy environments are made
difficult by imperfect and general-purpose/heavyweight
tools/frameworks and the actual business problem to solve is simple.
Even Hibernate and Spring are not necessarily the road to heaven.
So what are your ideas about using STM? What do you think the provided
value could be?
Tom Davies
RDBs, the inevitable abstraction leaks makes maintaining your data
model via ORM expensive in development time compared to a non-
persistent data model (obviously :-))
I'd be interested to see whether STM+persistence could make a
persistent data model more like a non-persistent data model from a
development perspective.
Prevayler is an example of an approach which does away with any
persistence mapping, but (IMHO) it has two flaws. First, only one
thread can update at a time, and second, your entire data model and
all the indexes you need to access your data must fit on the heap at
the same time. For many applications I think these restrictions are
not acceptable.
I imagine an STM+persistence system using mutable data structures
which allow a high degree of concurrent modification without conflict
(HashMaps, Skiplists?) and an LRU cache of references. The values
stored in the mutable data structures would be immutable.
Of course, your working set is still going to need to be on the heap,
including the indexes you traverse.
I imagine that when a transaction completes, an atomic write would be
queued (write-behind) which would write the new values of all the refs
updated in the tx to the store. You could have a single thread which
wrote the transactions.
A Ref could be in a loaded or empty state -- when an empty Ref is
read, its value would be loaded from the store (or from a committed tx
waiting to write). Refs could hold their contents with weak references
so that they could be garbage collected as needed, or some more
sophisticated LRU scheme could be used.
Peter Veentjer
> While Hibernate and other ORM tools are usually a good way to use
> RDBs, the inevitable abstraction leaks makes maintaining your data
> model via ORM expensive in development time compared to a non-
> persistent data model (obviously :-))
Yes. Another thing that often was more complex than it needed to be is
concurrency control on the database itself (people not understanding
isolation levels or how they are implemented on the database). Causing
all kinds of locking/race problems and in some cases complicated application
architecture to 'fix' these issues.
So I want to have easy control on optimistic and pessimistic locking.
Another thing that always annoyed me is that database don't support blocking
operations. So if you need to implement some queue (so pick up
records that are in a certain state) you need to add some scheduler
(quartz often)
that does the polling.
Quartz is certainly not a bad product... but doing a:
workQueue.take(...)
is way simpler.
> I'd be interested to see whether STM+persistence could make a
> persistent data model more like a non-persistent data model from a
> development perspective.
Same here. I'm already playing with a small storage engine, but in the
0.5 release I want to introduce an experimental one that can be integrated
with the AlphaStm. Nice for experimentation purposes and to get a feel of
what works.
> Prevayler is an example of an approach which does away with any
> persistence mapping, but (IMHO) it has two flaws. First, only one
> thread can update at a time, and second, your entire data model and
> all the indexes you need to access your data must fit on the heap at
> the same time. For many applications I think these restrictions are
> not acceptable.
Strange restrictions.. I'll make sure that Multiverse doesn't suffer from those.
> I imagine an STM+persistence system using mutable data structures
> which allow a high degree of concurrent modification without conflict
> (HashMaps, Skiplists?) and an LRU cache of references. The values
> stored in the mutable data structures would be immutable.
>
> Of course, your working set is still going to need to be on the heap,
> including the indexes you traverse.
>
> I imagine that when a transaction completes, an atomic write would be
> queued (write-behind)
I'm going to make write-through/write-behind configurable.
> which would write the new values of all the refs
> updated in the tx to the store. You could have a single thread which
> wrote the transactions.
>
> A Ref could be in a loaded or empty state -- when an empty Ref is
> read, its value would be loaded from the store (or from a committed tx
> waiting to write). Refs could hold their contents with weak references
> so that they could be garbage collected as needed, or some more
> sophisticated LRU scheme could be used.
I'm not too worried about that part. I'm more worried about lazy loading and
the impact on performance. I don't to make each and every slower because
of additional checks since each object should be able to persist.
> --
> You received this message because you are subscribed to the Google Groups "multiverse" group.
> To post to this group, send email to googlemu...@googlegroups.com.
> To unsubscribe from this group, send email to googlemultiver...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/googlemultiverse?hl=en.
>
>
Peter Veentjer
programming at my work
and continuing on Multiverse at home for the last 7 hours.