Fork/Join criticism

[Sander Mak]

Hi all,

I've been digging around about in the whole fork/join framework since it was included with JDK7. Nothing really new, but seemed like a good time to dig into the specifics. I ran into this article/rant: http://www.coopsoft.com/ar/CalamityArticle.html

Now, obviously, this guy has a biased viewpoint, having written a 'Fork Join Conqueror' (http://coopsoft.com/ar/ConquerArticle.html) and all... Still, he seems to raise some valid points it seems. Anyone care to discuss/debunk his arguments?

Cheers,

Sander

[Reinier Zwitserloot]

I can retort a few of these, but certainly not all of it.

--Exceedingly Complex--

Yes, but, complaining that F/J (ab)uses com.sun.misc.Unsafe is just stupid, in my opinion. The whole point of F/J is that _it_ abstracts away the considerable complexity inherent in aggressively efficient multicore usage so that the code written using the F/J framework is simple. Also, a framework like F/J is one of the few times you actually get to adopt the maxim "Micro-optimize this stuff to pieces, I don't care if the code becomes far more difficult to manage going forward". That's what core frameworks have to do to get the job done. For example, quite a few code in Math is very, lets be nice and say 'uniquely' written, i.e, complete gobbledygook with a comment 9x as large as the method explaining the arcane voodoo magicks going on, but, nobody should be complaining about this. Math is complex so your code doesn't have to be.

Why is F/J such as a mess of inheritance? Yeah, well, I kinda agree with the ranter on that one. inheritance isn't, as far as I know, a clear win for performance (the opposite, usually), and I doubt it makes the API any cleaner either.

--Unstable--

We have but the word of our ranter. The fact that F/J has a long history of tickets, assuming they've all been fixed, doesn't really say much about how unstable this piece of software is. All software has bugs, a project that carefully maintains a list and is open to 3rd party tickets shouldn't be punished for that. Let's flip it around: If F/J did not have an extensive bug list, would that mean it IS stable? That's clearly a silly argument.

Multicore is inherently complicated, though outsiders usually think "Geez, this has got to be much, much simpler than these supposed rocket scientists are making it!". Guess what? Only outsiders hold that view. Once you tussle with trying to write efficient multi-core code that works on all platforms and acts consistently, you start to agree that it's just that complicated, period.

--Lack of attributes--

A lot of these I just don't follow at all. What does this mean?: "There is no error detection/recovery/reporting (other then thrown exceptions)"? What kind of detection/recovery/reporting should F/J be doing? It's core CPU work, errors at this level are invariably unrecoverable and highly unexpected (your CPU or OS is buggy, or the framework is), unless the cause is a bug in hosted code, but propagating exceptions back out of the fork/join workers is exactly how a library is supposed to handle this, so.... what's the complaint again? I don't understand.

Stall detection, anomaly detection, and performance monitoring is nice, but, I'm kind of at a loss in regards to the flow of the rant. First the argument is that this should be simpler, then the argument turns into: But it's missing all these possible but quite complicated features. Which way is it?

--Academic exercise--

This stuff is beyond my comprehension, unfortunately.

--Inadequate in scope--

F/J working on either J2ME or an EE virtual server buit around RMIing calls around the server park? Come on, that's a preposterous idea. It would be nice, but, so are unicorns.

Didn't feel like going further into it today.

[Kirk]

Hi,

I can only comment on the use of Unsafe and inheritance and performance. There are cases, and F/J is most likely one of them, where inheritance could be a win. Getting separation putting memory distance between shared an non-shared variables can result in them being place in difference cache lines. That can have the effect of prevent multi-thread access from causes each thread to cause the other threads to continuously flush their cache. Cache flushing can be a huge drain on performance.

Until we get a normalized path to CAS that is baked into the JMM or Java Language Specification, we're stuck with unsafe.

Regards,

Kirk

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.
To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/J22idazMQUEJ.
To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[ricky.c...@gmail.com]

I haven't followed this very well but can you explain how inheritance makes F/J not cause cache flushes?

Sent from my BlackBerry® wireless device

---

From: Kirk <kirk.pe...@gmail.com>

Sender: java...@googlegroups.com

Date: Fri, 21 Oct 2011 10:41:51 +0200

To: Ray Hindman<java...@googlegroups.com>

ReplyTo: java...@googlegroups.com

Subject: Re: [The Java Posse] Re: Fork/Join criticism

[Jess Holle]

The alternative to inheritance is general object composition -- which allows each of the objects to be strewn all of the heap with respect to one another.

Inheritance, however, stuffs all the fields into 1 bucket that all moves together.

Object fields are pointers to elsewhere in the heap in any case, of course.  I'm talking about primitives.

[Edward Harned]

I wrote the articles and the software for fork-join. I've been doing fork-join since the mid 1980's so I know a bit about how to do it.

Doug Lea is a professor. He developed an application service. The two often don't mix.

If you take a careful, comprehensive look at the code you'll see that it is more about impressing his peers then about producing an application service that is both efficient and maintainable.

There are ten points in the calamity article. Taken together they show that the F/J framework does not belong as part of core Java. If he wants to have a private package, even part of the extensions framework, then so what? The article was not a critique of anyone, but of the framework's place as the core of all Java development.

I wrote a divide-and-conquer framework without using the Unsafe class. It is faster than the Java 7 versions. It is flexible, simple, and documented fully. There are differences between system programming and application programming. If Doug Lea wanted a system program to live inside the JVM, then he should have built one. You can download my software and compare the two. Then perhaps you'll be better able to comment on efficient, complex, etc.

Ed

[Kirk]

putting distance between immutable fields and shared mutable fields causes them to show up in difference cache lines. Reads from different threads won't cause cache line flushing.

Regards,

Kirk

[Cédric Beust ♔]

On Fri, Oct 21, 2011 at 6:53 AM, Edward Harned <edha...@gmail.com> wrote:

If you take a careful, comprehensive look at the code you'll see that it is more about impressing his peers

Without passing judgment on the technical value of your article, I find this claim particularly offensive toward Doug Lea considering his history and contributions to the Java platform for the past fifteen years.

-- 

Cédric

[ricky.c...@gmail.com]

With the way you put that I can't imagine why nobody's listened to you before.

Sent from my BlackBerry® wireless device

---

From: Edward Harned <edha...@gmail.com>

Sender: java...@googlegroups.com

Date: Fri, 21 Oct 2011 06:53:01 -0700 (PDT)

To: <java...@googlegroups.com>

ReplyTo: java...@googlegroups.com

Subject: [The Java Posse] Re: Fork/Join criticism

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/An7pbSTlXq4J.

[Sander Mak]

Edward,

I agree with the others that a little courtesy goes a long way, even more so if you're trying to persuade people. 

Okay, with that out of the way, on to the subject matter. I did not expect the author to reply on this forum... but since you are here: could you explain what you mean with the following observation in your article? 'Since the internal structure is so entangled with the calling client, there is almost no way to alter control variables'

Thanks,

Sander

[Russel Winder]

On Fri, 2011-10-21 at 17:11 +0000, ricky.c...@gmail.com wrote:
> With the way you put that I can't imagine why nobody's listened to you before.

A paradigm of irony. Excellent.

--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel...@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@russel.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder

[Sander Mak]

Hi Reinier,

Thanks for your long analysis, much appreciated. I partly agree with you on the complexity part, as far as the implementation goes. However the design of the framework being too complex is not as easily forgivable as the implementation of a single Math method being to complex IMO. The former is more or less exposed to application programmers (class hierarchies in public Javadoc etc.), whereas the latter is just a detail nobody bothers with.

Some of the arguments were a bit beyond my comprehension as well, that's why I started this discussion :) 

Thanks,

Sander

[opinali]

This debate can be approached in many ways, I could also look in depth at each point of your article... but I'll propose a different rebuttal: first, you realize that all previous work of JSR-133 and JSR-166, i.e. the JMM and the whole of java.util.concurrent, was designed and implemented by much the same people (not just Doug) and shares many of the same traits - for one thing, j.u.c is a quite big and complex library, its implementation has all the things you hate like very complex code and intense reliance on Unsafe tricks, lots of the complexity are exposed in the public API etc. etc.

Now, you admit that all that previous work is considered a MASSIVE success? I, for one, have long enjoyed the power of the Java5+ concurrency APIs and features; they result in application code that is extremely powerful, scalable, maintainable, robust, flexible... AND simple. (At least compared to any pre-Java5 alternatives... if you can do even better, hats off to you, but it doesn't change the fact that JSRs 133/166 were awesome.) And of course it's not just me; this is a well-known success case of the Java platform.

Now you come to us and tell that FJ, which is basically the next chapter of the same book, written by the same authors and basically in the same style, is some piece of junk. Oh and by the way you authored a competing book. Do you see how difficult is to give you any credibility?

And just one more thing: yes FJ's implementation can be still improved in some places - and improved it will be. For one thing, CPU affinity is an obvious issue/opportunity, and I have no doubt that the implementation will eventually take advantage of that. This doesn't even need to wait for a public API, Oracle can introduce internal APIs to obtain and control affinity - just like it does for many other things (e.g. NUMA optimizations for memory management and GC). This can even happen in maintenance updates, no need with wait for JDK.Next. Although of course I'd like to have a public API too some day, but that will need to wait the next JDK.

A+
Osvaldo

[Cédric Beust ♔]

On Fri, Oct 21, 2011 at 6:13 PM, opinali <opi...@gmail.com> wrote:

Now you come to us and tell that FJ, which is basically the next chapter of the same book, written by the same authors and basically in the same style, is some piece of junk. Oh and by the way you authored a competing book.

He also authored a competing framework, which he recommends as the only alternative to Fork/Join at the end of his article.

This is what I find pretty ironic: one of the good things in academia research, which he mocks in his email earlier, is that publications of this kind are usually followed by a whole list of references that have to be diverse and cover all kinds of topics by a lot of various authors.

Maybe he could learn a thing or two from this style of research instead of deriding it.

-- 

Cédric

[Simon Ochsenreither]

While I have no opinion on the the proposed "alternative" I think he is right in one case.

The popularity is in no way a measurement of quality and the JDK suffers pretty much from the approach of throwing things into it which are "good enough", but not "the best thing possible".

While often "good enough" is the best approach, it is not acceptable for a standard library.

Bye,

Simon

[opinali]

On Saturday, October 22, 2011 12:43:17 AM UTC-4, Cédric Beust ♔ wrote:

On Fri, Oct 21, 2011 at 6:13 PM, opinali <opi...@gmail.com> wrote:

Now you come to us and tell that FJ, which is basically the next chapter of the same book, written by the same authors and basically in the same style, is some piece of junk. Oh and by the way you authored a competing book.

He also authored a competing framework, which he recommends as the only alternative to Fork/Join at the end of his article.

Well, above I was using "book" as a metaphor for "framework" :) but it's also nice to remember JCIP, which is *THE* modern book about applied/practical concurrent programming - if anybody says that book is also "academic", it's crazy talk.

The single serious criticism I've ever heard about j.u.c. and the continuing work of JSR-166, is that it furthers the paradigm of shared-memory concurrency. But, like it or not, it's the appropriate thing to do at the level of the JavaSE platform and Java Language. Then you can use these as [extremely good] building blocks for higher-level languages and concurrency frameworks, like Scala+Actors etc. And j.u.c. is well designed to accommodate future innovation at the runtime or language levels - from Java syntax sugar a la C# 5, to lambdas, coroutines, affinnity..., all these things will be easy to fit, indeed they have been considered in the design.

A+
Osvaldo

[Russel Winder]

On Sat, 2011-10-22 at 04:32 -0700, opinali wrote:
[ . . . ]

> The single serious criticism I've ever heard about j.u.c. and the continuing
> work of JSR-166, is that it furthers the paradigm of shared-memory
> concurrency. But, like it or not, it's the appropriate thing to do at the
> level of the JavaSE platform and Java Language. Then you can use these as
> [extremely good] building blocks for higher-level languages and concurrency
> frameworks, like Scala+Actors etc. And j.u.c. is well designed to
> accommodate future innovation at the runtime or language levels - from Java
> syntax sugar a la C# 5, to lambdas, coroutines, affinnity..., all these
> things will be easy to fit, indeed they have been considered in the design.

Well here we have a real bone of contention. Java reified shared memory
multithreading in the early 1990s. It was an error then and is certainly
an error now -- at least as being pushed as the "one true way" of
concurrency since it makes parallelism very hard. Why should it remain
the "one true way" in Java? It is this closed mindset that is a big
problem with Java. Java should have actor libraries, dataflow
libraries, CSP, that applications folk use. Certainly shared memory
multithreading is an important framework level technique, but no
applications programmer should ever have to use such a low level model
of concurrency and parallelism.

GPars (http://gpars.codehaus.org) brings all of these things to Groovy
and Java programmers. JCSP is a CSP framework for Java. DataRush is a
commercial offering from Pervasive that offers dataflow. These models
thrash explicit use of shared memory multithreading for applications
development.

Please can we ditch the idea that Java means using shared memory
multithreading and that to use other models we have to use other
languages. It is not true and it is damaging in the face of ubiquitous
parallel hardware.

[Kirk]

On Oct 22, 2011, at 1:32 PM, opinali wrote:

On Saturday, October 22, 2011 12:43:17 AM UTC-4, Cédric Beust ♔ wrote:

On Fri, Oct 21, 2011 at 6:13 PM, opinali <opi...@gmail.com> wrote:

Now you come to us and tell that FJ, which is basically the next chapter of the same book, written by the same authors and basically in the same style, is some piece of junk. Oh and by the way you authored a competing book.

He also authored a competing framework, which he recommends as the only alternative to Fork/Join at the end of his article.

Well, above I was using "book" as a metaphor for "framework" :) but it's also nice to remember JCIP, which is *THE* modern book about applied/practical concurrent programming - if anybody says that book is also "academic", it's crazy talk.

The single serious criticism I've ever heard about j.u.c. and the continuing work of JSR-166, is that it furthers the paradigm of shared-memory concurrency. But, like it or not, it's the appropriate thing to do at the level of the JavaSE platform and Java Language.

It is an appropriate thing to do if you understand impacts on hardware and dynamic vs static (one time assignment models) impacts on scalability.

Regards,

Kirk

[Kirk]

On Oct 22, 2011, at 1:46 PM, Russel Winder wrote:

> On Sat, 2011-10-22 at 04:32 -0700, opinali wrote:
> [ . . . ]
>> The single serious criticism I've ever heard about j.u.c. and the continuing
>> work of JSR-166, is that it furthers the paradigm of shared-memory
>> concurrency. But, like it or not, it's the appropriate thing to do at the
>> level of the JavaSE platform and Java Language. Then you can use these as
>> [extremely good] building blocks for higher-level languages and concurrency
>> frameworks, like Scala+Actors etc. And j.u.c. is well designed to
>> accommodate future innovation at the runtime or language levels - from Java
>> syntax sugar a la C# 5, to lambdas, coroutines, affinnity..., all these
>> things will be easy to fit, indeed they have been considered in the design.
>
> Well here we have a real bone of contention. Java reified shared memory
> multithreading in the early 1990s.

I disagree, to use one model in exclusion to all others is dogma. Java supports mutability but it also nicely supports other techniques.

Regards,
Kirk

[Edward Harned]

A lot to reply to:

This article is not about Doug Lea. It is about the application service he wrote. Doug Lea is a fine scientist, educator, etc. Application programming is not his forte. The majority of code in the F/J framework is not from other but is one person. I love the JSR166 work. Use it all the time. Most of those classes are what is known as system programming. They are API's. The F/J framework is an application program. It belongs in the extensions framework, perhaps as javax.parallel. It is not the next chapter in the same book. It is a different book.

>Since the internal structure is so entangled with the calling client, there is almost no way to alter control variables

The client code imbeds the framework in the initial call as a parameter. The framework then calls the client to do the processing. That is simple to program. But there is no way to get at the framework to change number of threads to use per call etc. A much better way is to separate the client call, server processing, and actual computation. Now the server is separately addressable to monitor performance, alter variables and save statistics.

I had many conversations with Doug Lea last year. I did a proof of concept showing him that scatter-gather was a far superior algorithm than work-stealing. Just a simple load balancing increased throughput 5-7 times. Also separating the client from the server allowed better tuning (as above.) He pretty much ignored what I said. Eventually, I built the divide-and-conquer version of Tymeac from the embarrassingly parallel version. Now developers have a choice.

The real test of scalability and performance comes when using these frameworks for applications that generate hundreds of thousands and millions of Tasks (like Scala.) Using work-stealing where Tasks all go into the same queue from which they were spawned and other threads have to go looking for work, is not the best approach.

Let's be honest here. I did not mock academic research. Research and building on prior research is the scientific method. The problem is that research on work-stealing is for operating systems controlling work on CPU's. Application programming is a whole different beast. There is no academic research on work-stealing outside a controlled environment. Cilk and jCilk use a compiler and run time to control Tasks.

Ed

 

[Edward Harned]

I missed addressing the Unsafe class usage. First of all, why do you think they call it Unsafe? If you’re not familiar with the Class, then go look at it.

The Unsafe class is primarily for profilers and debuggers. They need direct access to memory. The concurrency utilities use Unsafe to get around the limits of wait()/notify() using park() and unpark(). There may also be an issue with the atomic instructions on CPUs. Most processors support atomic operation like FetchAndAdd and CompareAndSwap, but not in the same way. Some use LL/SC instead of CAS. So there is surely a need today for using Unsafe for atomic operations. API’s do things normal application programs don’t so it is what it is.

The F/J framework is not an API. It is an application program. That is the major difference. There is absolutely no need to program in pseudo-C to do fork-join. Here is a quote from the ForkJoinWorkerThread:
“Efficient implementation of these algorithms currently relies on an uncomfortable amount of "Unsafe" mechanics.”

Even the author knows he’s overdone it. He doesn’t use a standard deque from the concurrency utilities, he hard codes direct memory access to emulate a deque. The code in all his classes have massive amounts of Unsafe use. Why? Is it to get around the slow, clumsy work-stealing algorithm? I don’t see a need for it. It adds a huge amount of complexity. I can say from decades of experience, complexity always fails, always.

I developed four fork-join frameworks that run on Java SE, ME, and Android without any Unsafe usage. They’re faster than the F/J framework and have professional attributes. Unsafe has no place in application programming.

[Simon Ochsenreither]

I would love to see an implementation backing Scala's parallel collections as a proof of concept.
It should be even possible to have it as drop-in replacement because the parallelization mechanics are not exposed to the user.

What do you think? Would love to see some performance numbers ...

[Kirk]

I really don't get your objection to unsafe. It's more like hand waving than some real concern. It gives access to instructions and until we have a Java level abstraction that is safe, I don't see why access should be taken away. Debuggers and profilers will use the JVMTI and with that you have access to everything without having to play games with pointers in Java.

Kind regards,

Kirk Pepperdine

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/6a9PfIokayIJ.

[opinali]

On Saturday, October 22, 2011 7:46:40 AM UTC-4, Russel wrote:

On Sat, 2011-10-22 at 04:32 -0700, opinali wrote:
[ . . . ]
> The single serious criticism I've ever heard about j.u.c. and the continuing
> work of JSR-166, is that it furthers the paradigm of shared-memory

> concurrency. But, like it or not, it's the appropriate thing to do at the (...)

Well here we have a real bone of contention.  Java reified shared memory
multithreading in the early 1990s. It was an error then and is certainly

We don't need to debate this -- the past is the past, and like it not this is the paradigm of concurrency embedded in Java and in billions of existing LOC of Java apps, this this must be supported forever and also improved continuously in the extent that's possible. I don't claim that we cannot have a new concurrency model as an additional option (hi @Kirk), I just say that the existing system must keep moving'. Deprecating the shared-memory concurrency model is not realistic - it's not like, say, deprecating entity beans for JPA or deprecating Swing for JavaFX; it's not a single library that is only used by a subset of applications, it's a core feature that is used directly or indirectly by 100% of all code (apps, core libs, third-party libs... including the implementation of alternative languages and concurrency frameworks). Even the introduction of j.u.c. in JDK 5 was not a radical break with the past; I for one, had significant legacy code written with straight threads/monitors or with other libs (like Apache commons-pool) that I was able to quickly review to benefit from j.u.c.
 

an error now -- at least as being pushed as the "one true way" of
concurrency since it makes parallelism very hard.  Why should it remain
the "one true way" in Java?  It is this closed mindset that is a big
problem with Java.  Java should have actor libraries, dataflow
libraries, CSP, that applications folk use.  Certainly shared memory

I agree, but I think this kind of innovation already happens in the right places - the JVM-languages communities (Clojure, Scala etc.). We Java-lang diehards may complain about learning and adopting a new language as precondition for a new concurrency model (for one, I'm envious of Clojure's persistent collections), but that's the only *good* way to do it, because Java lacks even the slightest amount of support for these other models. For one thing, there's no real support for immutability ('final' is not good enough, we need full "constness" support a la C++).

Something like an actor library could be added to core Java, but

1) who would shoulder the effort? Oracle is busy enough improving the core paradigm (as it should), including hard low-level optimizations like lock elision etc. Concurrency is not everything, there's a ton of other big-ticket RFEs that are pet ideas of many hackers, should Oracle do them all?
2) and even assuming that Oracle has the resources to implement all our pet RFEs, should all of them be thrown into the JRE? Many people already complain of bloat when APIs that *are* used by a massive number of apps, like XML / Web Services stuff, are added to the core...
3) so maybe they add actors, only to make 50% of the community pissed of that they didn't pick some other next-gen concurrency idea, such as STM?
4) what to do when people start claiming for language syntax support for the new concurrency system? Add tons of extra syntax on top of the already big and hard-to-evolve Java language?

A+
Osvaldo
 

multithreading is an important framework level technique, but no
applications programmer should ever have to use such a low level model
of concurrency and parallelism.

GPars (http://gpars.codehaus.org) brings all of these things to Groovy
and Java programmers.  JCSP is a CSP framework for Java.  DataRush is a
commercial offering from Pervasive that offers dataflow.  These models
thrash explicit use of shared memory multithreading for applications
development.

Please can we ditch the idea that Java means using shared memory
multithreading and that to use other models we have to use other
languages.  It is not true and it is damaging in the face of ubiquitous
parallel hardware.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russ...@ekiga.net

[opinali]

On Saturday, October 22, 2011 10:11:21 AM UTC-4, Edward Harned wrote:

This article is not about Doug Lea. It is about the application service he wrote. Doug Lea is a fine scientist, educator, etc. Application programming is not his forte. The majority of code in

You mean that Doug (plus JSR166 contributors) wrote FJ from the top of their ivory tower, without studying application behavior, without feedback from early-adopters etc.? You don't have to be a professional Chef in order to provide great veggies or meat; you only need to be a good farmer (and listen to some Chefs, read their recipes books, study their menus...).
 

the F/J framework is not from other but is one person. I love the JSR166 work. Use it all the time. Most of those classes are what is known as system programming. They are API's. The F/J framework is an application program. It belongs in the extensions framework, perhaps as javax.parallel. It is not the next chapter in the same book. It is a different book.

This categorization of FJ as an "application program" is fallacious. If I have some app code that can use FJ, how exactly isn't FJ a conventional API? I would even say that FJ is a relatively low-level API, that will be often useful to build higher-level APIs (e.g. some BLAS, or image processing library) on top of it.

You make some specific criticism to FJ's API and design, and they may have merit. But why didn't you actively participate of the concurrency-interest list and JSR166 discussions (that were completely open - no need to even be part of the JSR EG)? Sorry, private talk with Doug Lea doesn't count, if you are not willing to debate your opinion more openly with the general Java concurrency community. I can find two relevant messages from you in the OpenJDK lists: one from Dec 2010 with the initial claim that FJ was no good and a pointer to an early article, another from Feb 2011 promoting the full Calamity article. Even the first message was too late, FJ was long a done deal, no wonder neither message deserved a single reply.

Your claims of 5-7X better throughput are just crazy talk until you show off the benchmark code, allow the community to validate if this doesn't suffer of benchmark flaws, including biased selection of problems / scenarios.

And a final thing, I see that jCilk extends Java, it has some kind of preprocessor. If you need to do this only to get some cool syntax sugar to avoid verbose API invocations, fine. But if you need it because the jCilk-to-Java compiler makes some important transformations / optimizations that are essential for your concurrency system... then good luck with that... you would be better off by joining forces with some other language that is better designed for concurrency (like Clojure/Scala/Fantom) or that embraces DSLs (like Groovy).

A+
Osvaldo

[ricky.c...@gmail.com]

Const always crops up, but const doesn't really help except in single-threaded programs, because all const says is that "I'm not going to modify this". Whichever other thread that has a non-const reference can still pull the rug from underneath us.

Sent from my BlackBerry® wireless device

---

From: opinali <opi...@gmail.com>

Sender: java...@googlegroups.com

Date: Sun, 23 Oct 2011 05:02:57 -0700 (PDT)

To: <java...@googlegroups.com>

ReplyTo: java...@googlegroups.com

Subject: Re: Models of Parallelism [ was Re: [The Java Posse] Re: Fork/Join criticism ]

--
You received this message because you are subscribed to the Google Groups "The Java Posse" group.

To view this discussion on the web visit https://groups.google.com/d/msg/javaposse/-/DPdstDGotp4J.

[Kevin Wright]

Of course, you can always use Akka within a Java program.

Yes. Akka was written in Scala, and is arguably best used from Scala (the language has many constructs that will help you out). But it's not the only way, the guys have gone to great lengths to ensure that it also provides a rather nice Java API:

http://akka.io/docs/akka/1.2/java/index.html

That buys you actors, dataflow, STM, etc. A veritable smorgasbord of concurrency techniques! What's not to like?

[Russel Winder]

On Sun, 2011-10-23 at 17:31 +0100, Kevin Wright wrote:
> Of course, you can always use Akka within a Java program.
>
> Yes. Akka was written in Scala, and is arguably best used from Scala (the
> language has many constructs that will help you out). But it's not the only
> way, the guys have gone to great lengths to ensure that it also provides a
> rather nice Java API:
>
> http://akka.io/docs/akka/1.2/java/index.html
>
> That buys you actors, dataflow, STM, etc. A veritable smorgasbord of
> concurrency techniques! What's not to like?

Because GPars (http://gpars.codehaus.org) is better?

(OK so I have a vested interest in promoting GPars as the concurrency
and parallelism framework for Java and Groovy. At some point I'll have
to do a proper benchmark comparison between Akka and GPars, but till
then I'll just stick with unsupported prejudice :-)

--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel...@ekiga.net

[Edward Harned]

opinali:

Good point about not participating in JSR166. I wasn't aware of just what DL was doing. From the Brian Goetz article "Stick a fork in it" the framework was supposed to be for compute only on very large arrays. More of the scientific/academic stuff. I do business application so I wasn't concerned. Once I found out last summer exactly what it included, yes it was too late.

Benchmarks can be manipulated in any direction which is why I said you need to consider the internal workings.

jCilk is a separate application framework; exactly where parallelism belongs.

Kirk:

Unsafe direct memory modification bypasses all the managed runtime safety guarantees and security features of the Java language. No bounds checks. No memory overwrite checks, etc. And it adds unnecessary complexity. As far as I can see there is not other reason for using it in the F/J framework other then to get faster execution. There is a j.u.c.LinkedBlockingDeque so there was no need to invent another.

Ed

[Simon Ochsenreither]

Groovy is still alive?

*shrugs*

:-D

[Kevin Wright]

Sure it is

Too many big corps (e.g. Oracle, SpringSource) have thrown to much behind the lang to allow it to die out simply because mere *developers* don't think it has much value nowadays.

Nor is it especially Groovy's fault that it got superseded so quickly and comprehensively.  James Strachan created something quite elegant when compared to Java, in absence of any other alternatives.

[Cédric Beust ♔]

On Sun, Oct 23, 2011 at 12:26 PM, Kevin Wright <kev.lee...@gmail.com> wrote:

Sure it is

Too many big corps (e.g. Oracle, SpringSource) have thrown to much behind the lang to allow it to die out simply because mere *developers* don't think it has much value nowadays.

You make it sound as if the only reason why Groovy is still around is because of some corporate backing, but 1) Groovy has had pretty much zero corporate backing and 2) Groovy became popular on its own merits, and I see it used in a lot of places around me (much more than Scala, for what it's worth).

 

Nor is it especially Groovy's fault that it got superseded so quickly and comprehensively.

It's the second most popular language on the JVM, so I'm not sure why you think it's been superseded (I'm not going to ask you which language you think superseded it, because we all know your answer).

-- 

Cédric

[Kirk]

If you get the unsafe stuff wrong it will throw reasonable exceptions at reasonable times. I know, I've done unreasonable things and was on the receiving end of some interesting exceptions ;-).

Every VM I've worked with (prior to Java) came with a backdoor (set of unsafe operations). The more famous one in SmallTalk was the method become:. Completely unsafe!! So the trend continues with Unsafe. Not to say that the needed operations should stay in Unsafe. I'd rather see them exposed in a safer way but to take away access to CAS (or the equivalent in IBM land) closes the door on some interesting solutions to concurreny. Look at Cliff Clicks coding style to higher concurrency. It relies on CAS and a number of other hardware/JMM features in a way that average programmers given todays tools should probably shy away from. That said, who are we to mother other developers away from experimenting with these techniques?

Regards,

Kirk

[Kevin Wright]

2011/10/23 Cédric Beust ♔ <ced...@beust.com>

On Sun, Oct 23, 2011 at 12:26 PM, Kevin Wright <kev.lee...@gmail.com> wrote:

Sure it is

Too many big corps (e.g. Oracle, SpringSource) have thrown to much behind the lang to allow it to die out simply because mere *developers* don't think it has much value nowadays.

You make it sound as if the only reason why Groovy is still around is because of some corporate backing, but 1) Groovy has had pretty much zero corporate backing and 2) Groovy became popular on its own merits, and I see it used in a lot of places around me (much more than Scala, for what it's worth). 

I search Google for "JDeveloper plugins", the 4th hit is the Groovy plugin.  I open up springsource.com and the second major link on that page is for Groovy & Grails.  This sure doesn't look like zero corporate backing to my eyes.

 

Nor is it especially Groovy's fault that it got superseded so quickly and comprehensively.

It's the second most popular language on the JVM, so I'm not sure why you think it's been superseded (I'm not going to ask you which language you think superseded it, because we all know your answer).

Bet you don't :)

If you just want a good dynamically-typed language on the JVM, JRuby and Clojure take the crown.  Either choice is richer and faster than Groovy.

If DSLs are your thing, JRuby and Scala are both good choices, with Scala taking the edge (just look at ScalaTest and Specs2, or how close ScalaFX looks to the original JavaFX, whilst still being based on JavaFX2).  Clojure is also very good at DSLs, so long as you're willing to concede that something can be domain-specific without demanding a syntax that's close to natural english.

*anything else* is better in terms of performance.  JRuby benefits from the many minor miracles that Charles Nutter seems to deliver on a daily basis.  Scala benefits from static typing, as does Mirah.  When released, I imagine that Kotlin and Ceylon will enjoy the same benefit, though Scala and Mirah will have enjoyed numerous rounds of optimisation by then.  

Even for small scripts Scala is now able to be specified via the "shebang" at the top of a script file, and startup time on recent scala releases is fast.

Arguably then, the only place in which Groovy can be considered to have an advantage is the current number of users and the availability of developers/books/etc.  That's fair conversation to have, it's just not one about technical merits.  Every single benefit that Groovy originally set out to offer over and above Java is now being offered, more so, by *at least* one other language.

[Paul King]

On Mon, Oct 24, 2011 at 6:36 AM, Kevin Wright <kev.lee...@gmail.com> wrote:
> [...snip...]  Every single benefit that Groovy originally set out

> to offer over and above Java is now being offered, more so, by *at least*
> one other language.

I think it is accurate to say that most of the features of Groovy
circa 2005 are now being offered in other languages. Yes, that's
probably true.

Cheers, Paul.

[clay]

That article is completely unreasonable. I'll say this:

This library is extremely fast and easy to evaluate if you are dealing
with the narrow class of divide/conquer processing tasks that this
relates to. It's a small API, only a handful of public classes, and
it's got a very narrow feature list. You can port any divide/conquer
task to this and test it out fairly easily.

I did quite a bit of benchmarking with quicksort and a minmax
algorithm. jsr166 was definitely fast. I thought I could beat it with
my own heavily optimized code; I was able to match it, but obviously,
it's great to have a high performance standardized API built into the
Java platform.

I'm heavily skeptical of the claims of 5-7x throughput increase, but
if you can suggest an alternative, I'd defintely evaluate it the next
time I'm facing one of these divide/conquer tasks.

[clay]

On the subject of high performance parallelism APIs, does anyone have
any thoughts on AMDs Aparapi (Java API to OpenCL hardware)?

[Edward Harned]

Clay:

You are right. The sort numbers are wrong.

I was comparing sorting a long array with sorting an object Long.
Instead of using ParallelLongArray I was using ParallelArray with object Long[].

Therefore, I adjusted the article number from 5-7 times faster to “almost 2 times faster.” Tymeac generally runs as well as and often faster then the JSR166 sort on a small numbers of processors. It all comes down to the task granularity and number of processors. The more real CPU’s to spread the work onto, the better load balancing works.

Ed