Parallelism and Concurrency
Casey Hawthorne
Parallelism - one task shared across multiple cores
Concurrency - different tasks shared among multiple cores
For both, the overhead of communication between the tasks compared to
the useful work done on the task itself, is the biggest challenge.
For parallelism, (and concurrency), any immutable data can be locally
stored, reducing memory latency, and since memory prices are falling
all the time, more and more local data can be accommodated.
So, for problems with many readers and only one writer, things are not
so bad.
For many writers and some readers, one wants as much immutable data as
possible, and also to localize the effects of mutable data, which
would seem to suit functional languages, like Haskell, which are
designed to "cleanly" separate pure code from side-effect code
(globally affecting code).
--
Regards,
Casey
Jon Harrop
> Parallelism and Concurrency: I would separate the two concepts:
> Parallelism - one task shared across multiple cores
> Concurrency - different tasks shared among multiple cores
No, you have defined data parallelism and task parallelism, respectively.
Definitions of concurrency differ but the general idea is that the control
flow of a concurrent program is non-deterministic.
> For both, the overhead of communication between the tasks compared to
> the useful work done on the task itself, is the biggest challenge.
I'd say the biggest challenge with concurrent programming is correctness
because non-deterministic control flow renders testing much less useful,
and the biggest challenge with parallel program is granularity.
> For parallelism, (and concurrency), any immutable data can be locally
> stored, reducing memory latency, and since memory prices are falling
> all the time, more and more local data can be accommodated.
Yes.
> So, for problems with many readers and only one writer, things are not
> so bad.
The readers must still get updates from the writer somehow.
> For many writers and some readers, one wants as much immutable data as
> possible, and also to localize the effects of mutable data, which
> would seem to suit functional languages, like Haskell, which are
> designed to "cleanly" separate pure code from side-effect code
> (globally affecting code).
There is a flip side: functional languages generate heaps with far more
pointers which bogs down the GC.
Moreover, the only reason to parallelize code is performance and Haskell is
a disaster in the context of performance.
--
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?u
Casey Hawthorne
>Moreover, the only reason to parallelize code is performance and Haskell is
>a disaster in the context of performance.
I take it you mean compared to OCaml.
Compared to other functional languages?
--
Regards,
Casey
Jon Harrop
>>Moreover, the only reason to parallelize code is performance and Haskell
>>is a disaster in the context of performance.
>
> I take it you mean compared to OCaml.
Compared to C, C++, Java, C#, F#, OCaml...
> Compared to other functional languages?
Do you mean purely functional?
Casey Hawthorne
parallelism, reminds me of the debates decades ago in AI whether
"intelligence" is in the program or the data.
We now know that the "intellegence" exhibited by the program is in
both.
I wonder if such a unification is possible in the data parallelism and
task parallelism worlds, which would lead to a better theory of
parallel architectures.
On Tue, 17 Nov 2009 02:26:52 +0000, Jon Harrop <j...@ffconsultancy.com>
wrote:
>Casey Hawthorne wrote:
>> Parallelism and Concurrency: I would separate the two concepts:
>> Parallelism - one task shared across multiple cores
>> Concurrency - different tasks shared among multiple cores
>
>No, you have defined data parallelism and task parallelism, respectively.
>
>Definitions of concurrency differ but the general idea is that the control
>flow of a concurrent program is non-deterministic.
>
--
Regards,
Casey
Jon Harrop
> I wonder if such a unification is possible in the data parallelism and
> task parallelism worlds, which would lead to a better theory of
> parallel architectures.
Well, functional programming unifies them because tasks are data. :-)
Jon Harrop
> Casey Hawthorne wrote:
>> I wonder if such a unification is possible in the data parallelism and
>> task parallelism worlds, which would lead to a better theory of
>> parallel architectures.
>
> Well, functional programming unifies them because tasks are data. :-)
Ok, not quite: tasks spawn more tasks but data doesn't (it can just be
nested). In fact, that is a good justification for my preferring task
parallelism to nested data parallelism. :-)