Clustered STM?

[Haddock]

Hello,

STM does really a great job in avoiding deadlocks and works fine when used in a single JVM. However, things are nowaydays often server-side and you end up with several servers for load balancing and clustering reasons. A good sample for this is Hazelcast. Enterprise archtitects simply demand that the application be clustered and then ScalaSTM cannot be used anymore.

Now I could implement my own remotely reachable ScalaSTM map. However, the difficulty would be to make it clustered (aka replicated). This seems to be a real hard problem to me and I can't work on something like that, because I have to spend my time earning my pay with application development ;-). So my question is whether clustered STM would be something the guys from Scala STM would consider thinking about how to do it or maybe even work on it.

Thanks, Haddock

[Ben Shapiro]

That seems like a recipe for awful performance. It sounds like what you want is one of the many cluster-friendly databases that are out there. Have you considered any of those options?

[Nathan Bronson]

Ben is right. STM algorithms are tightly coupled with the assumption that coherent reads are very fast and scalable. This is true on a single machine because coherence is baked into the CPU and its cache coherence protocols (MESI, ...), but across multiple nodes there is no longer a single coherent memory space. Once message passing is in software you are better off to use messages directly to implement your data store algorithm.

- Nathan

---
Typed with thumbs

On Mar 4, 2015 5:13 AM, "Ben Shapiro" <b...@getdown.org> wrote:

That seems like a recipe for awful performance. It sounds like what you want is one of the many cluster-friendly databases that are out there. Have you considered any of those options?

--

---
You received this message because you are subscribed to the Google Groups "Scala STM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-stm-expert-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Haddock]

It was just something that struck my mind. So I thought I should ask the experts. Thanks for the answers :-).

-- Haddock

[Haddock]

ScalaSTM came to my mind again ... ;-). All right, what about having a master and a slave where the master forwards all changes to the slave. Could that be implemented in a performant way? If so, is there some way in ScalaSTM to be notified about changes made by some commit that could then be forwarded to the slave?

Thanks, Haddock

Am Mittwoch, 4. März 2015 21:11:48 UTC+1 schrieb Nathan Bronson:

To unsubscribe from this group and stop receiving emails from it, send an email to scala-stm-expert-group+unsub...@googlegroups.com.

[Nathan Bronson]

Hi Haddock,

This is actually a super-hard problem for a couple of reasons. The first is that some sort of correspondence would have to be established between references in the two VMs. This is potentially solvable using some sort of object identifiers. The deeper and trickier problem is the parallelism gap between the master and the slave in this scenario. The master has complete flexibility over what order to commit, so many threads can try to commit in parallel and whoever wins wins. The slave, however, has to produce the same serialization order as the master. That means that the master either has to construct and convey information about transaction dependencies (which has a high cost in a fine-grained system like ScalaSTM) or the slave has to commit updates in a single-threaded fashion (making it very difficult to keep up with a multi-core master). Databases have the same problem. I don't know how viable the system is, but http://www.cs.cmu.edu/~dongz/papers/KuaFu.pdf seems to describe the challenge pretty well.

- Nathan

To unsubscribe from this group and stop receiving emails from it, send an email to scala-stm-expert-group+unsubscri...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Scala STM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-stm-expert-group+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Nathan Grasso Bronson
ngbr...@gmail.com

[Haddock]

All right, I see. Thanks :-)

Am Montag, 4. Dezember 2017 18:24:23 UTC+1 schrieb Nathan Bronson:

Hi Haddock,

This is actually a super-hard problem for a couple of reasons. The first is that some sort of correspondence would have to be established between references in the two VMs. This is potentially solvable using some sort of object identifiers. The deeper and trickier problem is the parallelism gap between the master and the slave in this scenario. The master has complete flexibility over what order to commit, so many threads can try to commit in parallel and whoever wins wins. The slave, however, has to produce the same serialization order as the master. That means that the master either has to construct and convey information about transaction dependencies (which has a high cost in a fine-grained system like ScalaSTM) or the slave has to commit updates in a single-threaded fashion (making it very difficult to keep up with a multi-core master). Databases have the same problem. I don't know how viable the system is, but http://www.cs.cmu.edu/~dongz/papers/KuaFu.pdf seems to describe the challenge pretty well.

- Nathan

On Mon, Dec 4, 2017 at 5:22 AM, Haddock <ffm...@web.de> wrote:

ScalaSTM came to my mind again ... ;-). All right, what about having a master and a slave where the master forwards all changes to the slave. Could that be implemented in a performant way? If so, is there some way in ScalaSTM to be notified about changes made by some commit that could then be forwarded to the slave?

Thanks, Haddock

Am Mittwoch, 4. März 2015 21:11:48 UTC+1 schrieb Nathan Bronson:

Ben is right. STM algorithms are tightly coupled with the assumption that coherent reads are very fast and scalable. This is true on a single machine because coherence is baked into the CPU and its cache coherence protocols (MESI, ...), but across multiple nodes there is no longer a single coherent memory space. Once message passing is in software you are better off to use messages directly to implement your data store algorithm.

- Nathan

---
Typed with thumbs

On Mar 4, 2015 5:13 AM, "Ben Shapiro" <b...@getdown.org> wrote:

That seems like a recipe for awful performance. It sounds like what you want is one of the many cluster-friendly databases that are out there. Have you considered any of those options?

--

---
You received this message because you are subscribed to the Google Groups "Scala STM Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-stm-expert-group+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "Scala STM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-stm-expert-group+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.