Parallel Programming in C++
Scott Robert Ladd
Is anyone doing parallel programming in C++?
I'm asking about true parallel programming for multiprocessor mainframes and
clusters; threads (light-weight processes) are something different (although
perhaps people need to know about them, too?)
What I'm looking for is some sense of current practice in developing
applications -- which technologies people are using (MPICH? Native
processes? Linux? Beowulf? Windows?) and the applications they're working
on.
My motivation? I've just finished a couple of parallel programming projects,
and I'm trying to decide what sort of articles to add to my web site (URL
below).
Thanks.
..Scott
--
Scott Robert Ladd
Coyote Gulch Productions -- http://www.coyotegulch.com
No ads. Just very free (and unusual) code and articles
Victor Bazarov
> Is anyone doing parallel programming in C++?
I am sure somebody does. But C++ does not have any specific
language elements to support parallel programming (except maybe
for reentrability of functions and 'volatile' specifier), which
makes conversations on this subject off-topic. Sorry. Maybe
you should try comp.parallel...
Victor
--
Please remove capital A's from my address when replying by mail
Ferdi Smit
running sunos) in C++ with the help of MPI, which actually is a C library.
Wrote some wrapper classes for MPI calls, worked like a charm.
Tho maybe it would've been easier in a specialized language, like Orca.
Scott Robert Ladd
> I am sure somebody does. But C++ does not have any specific
> language elements to support parallel programming (except maybe
> for reentrability of functions and 'volatile' specifier), which
> makes conversations on this subject off-topic. Sorry. Maybe
> you should try comp.parallel...
The language proper does not have parallel components, but it seems valid to
me to discuss parallel programm in C++ given the existence of C++ bindings
for OpenMP, MPI, and other parallel systems. I'm trying to figure out what
C++ programmers are doing with parallel programming; if I ask that in
comp.parallel, they'll tell me to go to a C++ group... ;)
Victor Bazarov
> > I am sure somebody does. But C++ does not have any specific
> > language elements to support parallel programming (except maybe
> > for reentrability of functions and 'volatile' specifier), which
> > makes conversations on this subject off-topic. Sorry. Maybe
> > you should try comp.parallel...
>
> The language proper does not have parallel components, but it seems valid
to
> me to discuss parallel programm in C++ given the existence of C++ bindings
> for OpenMP, MPI, and other parallel systems. I'm trying to figure out what
> C++ programmers are doing with parallel programming; if I ask that in
> comp.parallel, they'll tell me to go to a C++ group... ;)
We've been through that many times, haven't we? The language
proper doesn't have any network programming elements. But if
I ask about C++ in a network newsgroup, they will tell me...
This newsgroup is about the language, not about its applications
in different fields that happen to be of your or my interest today.
Please, don't get me wrong, parallel programming is an interesting
field, but it has nothing to do with the subject of this forum.
And you ought to know that, AFAICT, you're not here for the first
time.
As to parallel programming in a particular language, please submit
a proposal to split comp.parallel into comp.parallel.c++ and so on,
if that's what you want to discuss there. Or submit a proposal to
split comp.lang.c++ into comp.lang.c++.parallel, etc. And let's
see how those things go. Until then, you have nowhere to go, and
that's sad, I can relate (and many other folks probably can too).
But that's no reason to begin veering this newsgroup off its topic.
Edwin Robert Tisdale
> Is anyone doing parallel programming in C++?
Well, I guess that I am.
> I'm asking about true parallel programming
> for multiprocessor mainframes and clusters;
> threads (light-weight processes) are something different
> (although perhaps people need to know about them, too?)
>
> What I'm looking for is
> some sense of current practice in developing applications --
> which technologies people are using
> (MPICH? Native processes? Linux? Beowulf? Windows?)
> and the applications they're working on.
We use MPI (both MPIPro and MPICH).
We have three Linux clusters
including the original Beowulf cluster -- Hyglac.
We don't do Windows.
Our applications are data (image) processing and
numerical simulations.
We use both fast ethernet and Myrinet.
Most applications are SPMD.
Ferdi Smit
> a proposal to split comp.parallel into comp.parallel.c++ and so on,
> if that's what you want to discuss there. Or submit a proposal to
> split comp.lang.c++ into comp.lang.c++.parallel, etc. And let's
> see how those things go. Until then, you have nowhere to go, and
> that's sad, I can relate (and many other folks probably can too).
> But that's no reason to begin veering this newsgroup off its topic.
Submit a proposal to split another newsgroup so your slightly off topic
question can be posted there, and then hopefully be answered?? Bureaucracy
to it's fullest... It's so annoying that the last few days the only posts I
read are "Standard C++, the topic of this newsgroup, does not cover X. For
more information please refer to newsgroup Y". The number of those posts is
almost exceeding the number of OT posts.
Anyway, I understand and don't mean any offense, it just struck me as being
a little odd. Maybe it's related to having to come up with 50k forms for a
holiday visum haha :-)
Alexander Terekhov
>
> "Victor Bazarov" <vAba...@dAnai.com> wrote
> > I am sure somebody does. But C++ does not have any specific
> > language elements to support parallel programming (except maybe
> > for reentrability of functions and 'volatile' specifier), which
> > makes conversations on this subject off-topic.
That's total bullshit (your misconceptions w.r.t. *standard*
C/C++ 'volatiles' including), Victor.
> > Sorry.
'likewise'.
> > Maybe you should try comp.parallel...
Maybe.
> The language proper does not have parallel components, but it seems valid to
> me to discuss parallel programm in C++ given the existence of C++ bindings
> for OpenMP, MPI, and other parallel systems. I'm trying to figure out what
> C++ programmers are doing with parallel programming; if I ask that in
> comp.parallel, they'll tell me to go to a C++ group... ;)
Read this (ironically, YOU (Scott) WERE the one to whom I've
replied on comp.std.c++ -- perhaps you've just somehow missed
it there... NO PROBLEMS, it's just one click away from you ;-)):
http://groups.google.com/groups?selm=3CBF1465.48010582%40web.de
(Subject: Re: Boost.Threads and Dinkum CoreX)
"Scott Robert Ladd wrote:
>
> Hi,
>
> For true multiprocessing work on supercomputers and clusters, you can
find
> portable libraries like MPI (http://www.mpi-forum.org) and PVM
> (http://www.epm.ornl.gov/pvm/pvm_home.html). I've used MPI quite
> successfully on Beowulf clusters.
>
> Some compilers (Intel, for instance) support the OpenMP
> (http://www.openmp.org/) standard, which is a set of pragmas for
> implementing portable threading in C, C++, and Fortran.
Read this nice write-up on "hints" w.r.t. isolation/security/
performance/scalability/reliability/MPI/OpenMP/threads/processes/
IPC/coarse and fine grained parallelism, etc. (C and C++ mentioned
as well ;-)): ...."
And (since you've also mentioned some REAL stuff ("dinosaurs",
so to speak [I'm, personally, PROUD to have some "connections
with"]), perhaps, THIS ("COUPLING FACILITY" technologies, I
mean) too:
http://researchweb.watson.ibm.com/journal/sj/362/nick.html
(S/390 cluster technology: Parallel Sysplex)
regards,
alexander.
Victor Bazarov
> Scott Robert Ladd wrote:
> >
> > "Victor Bazarov" <vAba...@dAnai.com> wrote
> > > I am sure somebody does. But C++ does not have any specific
> > > language elements to support parallel programming (except maybe
> > > for reentrability of functions and 'volatile' specifier), which
> > > makes conversations on this subject off-topic.
>
> That's total bullshit (your misconceptions w.r.t. *standard*
> C/C++ 'volatiles' including), Victor.
>
> > > Sorry.
>
> 'likewise'.
Well... Thank you for explaining why what I wrote was "bullshit".
It was very insightful. And polite. What can I expect from
a fellow Russian (?) except an insult? If you have nothing to
say, maybe you shouldn't say anything? Think about it...
Alexander Terekhov
> "Alexander Terekhov" <tere...@web.de> wrote...
> > Scott Robert Ladd wrote:
> > >
> > > "Victor Bazarov" <vAba...@dAnai.com> wrote
> > > > I am sure somebody does. But C++ does not have any specific
> > > > language elements to support parallel programming (except maybe
> > > > for reentrability of functions and 'volatile' specifier), which
> > > > makes conversations on this subject off-topic.
> >
> > That's total bullshit (your misconceptions w.r.t. *standard*
> > C/C++ 'volatiles' including), Victor.
> >
> > > > Sorry.
> >
> > 'likewise'.
>
>
> Well... Thank you for explaining why what I wrote was "bullshit".
Well, OLL KORRECT -- I'll try to "explain":
A) Your 'conversations on this subject off-topic'-bullshit:
http://groups.google.com/groups?selm=1T7H8.24863%24YI5.411928%40twister.tampabay.rr.com
(Pay some attention to "Newsgroups: ....")
B) Your 'volatile'-bullshit:
http://groups.google.com/groups?threadm=a8hgtr%24euj%241%40news.hccnet.nl
(Subject: Hardware port)
http://groups.google.com/groups?selm=L9JR7.478%24BK1.14104%40news.cpqcorp.net
(Subject: Re: keyword volatile required ?)
Well, apart from your mysterious 'language elements' [please
explain!] for 'reentrability'... A + B ~= 'total bullshit',
or am I wrong?
> It was very insightful. And polite. What can I expect from
> a fellow
*Belo*
> Russian (?) except an insult?
Victor, I love you! You are great!! Is it better now? ;-)
> If you have nothing to say, maybe you shouldn't say anything?
Well, but you've just snipped all the 'insightful' stuff
I've said/pointed to!
regards,
alexander.
Victor Bazarov
> "Victor Bazarov" <vAba...@dAnai.com> wrote in message
news:<MXsH8.13934$ux5....@rwcrnsc51.ops.asp.att.net>...
> > "Alexander Terekhov" <tere...@web.de> wrote...
> > > Scott Robert Ladd wrote:
> > > >
> > > > "Victor Bazarov" <vAba...@dAnai.com> wrote
> > > > > I am sure somebody does. But C++ does not have any specific
> > > > > language elements to support parallel programming (except maybe
> > > > > for reentrability of functions and 'volatile' specifier), which
> > > > > makes conversations on this subject off-topic.
> > >
> > > That's total bullshit (your misconceptions w.r.t. *standard*
> > > C/C++ 'volatiles' including), Victor.
> > >
> > > > > Sorry.
> > >
> > > 'likewise'.
> >
> >
> > Well... Thank you for explaining why what I wrote was "bullshit".
>
> Well, OLL KORRECT -- I'll try to "explain":
>
> A) Your 'conversations on this subject off-topic'-bullshit:
>
>
http://groups.google.com/groups?selm=1T7H8.24863%24YI5.411928%40twister.tamp
abay.rr.com
> (Pay some attention to "Newsgroups: ....")
comp.lang.c++.moderated. Am I supposed to derive some knowledge
from that fact? Are you saying that the mere appearance of a post
on parallel programming in a different newsgroup makes it ALRIGHT
to post it here? I DON'T THINK SO.
> B) Your 'volatile'-bullshit:
>
> http://groups.google.com/groups?threadm=a8hgtr%24euj%241%40news.hccnet.nl
> (Subject: Hardware port)
>
>
http://groups.google.com/groups?selm=L9JR7.478%24BK1.14104%40news.cpqcorp.ne
t
> (Subject: Re: keyword volatile required ?)
So, those are links to prevoiusly conducted conversations on hardware
and threading. Both are irrelevant here, since they have nothing to
do with the subject at hand: parallel programming has no place in a C++
newsgroup. Or do you think otherwise? I said that C++ has no mechanisms
for parallel programming. I said that maybe, may be, 'volatile' and
'function reentrability' have some distant relevance, but nothing else.
You replied with "bullshit". Why? Probably because you're an MVP. I
can find no other explanation.
> Well, apart from your mysterious 'language elements' [please
> explain!] for 'reentrability'... A + B ~= 'total bullshit',
> or am I wrong?
You _are_ wrong. What the f*** does parallel programming have
to do with C++ language? Until proven otherwise, it _is_ off-topic
in comp.lang.c++. I pointed that out. You called my arguments
bullshit and offered NO counter-argument. I am still awaiting.
Either an explanation or an apology.
> > It was very insightful. And polite. What can I expect from
> > a fellow
>
> *Belo*
Oh, pardom me...
> > Russian (?) except an insult?
>
> Victor, I love you! You are great!! Is it better now? ;-)
Oh, much! I am all tears... Coming from you, it's so touching...
NOT!!! Just another attempt at an insult. Failed!
> > If you have nothing to say, maybe you shouldn't say anything?
>
> Well, but you've just snipped all the 'insightful' stuff
> I've said/pointed to!
What insightful stuff? You called my statement about irrelevance
of parallel programming to this newsgroup and my explanation why
it was so, "bullshit", while providing NOTHING to support such
characterisation. What did I miss? Please reread the entire
thread and tell me where I am mistaken. Or don't. Half-an-hour
from now I am not going to care...
Have a good Saturday.
Branimir Maksimovic
"Scott Robert Ladd" <sc...@coyotegulch.com> wrote in message
news:pv8H8.24872$YI5.4...@twister.tampabay.rr.com...
> Hi,
>
> Is anyone doing parallel programming in C++?
Everyone is doing parallel programming in C++.
Team of programmers are doing parallel programming in C++,
or any other programming language.
>
> I'm asking about true parallel programming for multiprocessor mainframes
and
> clusters; threads (light-weight processes) are something different
(although
> perhaps people need to know about them, too?)
If you think about distributing execution of single program accros several
CPU-s, (by single program I don't mean necessery SPMD model),
then same can be achived with threads, too (though one needs more
than one processor that share same memory to execute same algorithm
truly in parallel).
>
> What I'm looking for is some sense of current practice in developing
> applications -- which technologies people are using (MPICH? Native
> processes? Linux? Beowulf? Windows?) and the applications they're working
> on.
MPICH is implementation of MessagePassingInterace( which is well known
standardised API). I can't see how is that connected
with terms like processes, operating systems or particular computer
network configuration such as Beowulf?
>
> My motivation? I've just finished a couple of parallel programming
projects,
> and I'm trying to decide what sort of articles to add to my web site (URL
> below).
Well, first try to post to right newsgroups.
>
> Thanks.
>
> ..Scott
>
Greetings, Bane.
Alexander Terekhov
Branimir Maksimovic wrote:
[...]
> Well, first try to post to right newsgroups.
Bullshit.
What is the proper newsgroup to discuss, for example,
the stuff below?
http://www.terekhov.de/thread_ptr.txt
http://www.terekhov.de/mythread.c
http://groups.google.com/groups?selm=3CBA8C49.B0F40E4A%40web.de
Subject: Re: C++ and threads (invariant issues, actually)
Newsgroups: comp.std.c++
Date: 2002-04-15 18:50:29 PST
John Nagle wrote:
[...]
> C++ really does need language-level thread
> support, because there are so many language-level
> thread issues, of which this is only one. Yes,
> you can do everything with OS calls, but the
> language isn't giving you any help in getting it
> right. Are there any good designs for this being
> proposed?
FYI:
http://groups.google.com/groups?as_umsgid=3CA9EDE3.2227EE4A%40web.de
"....
The next step could be that "full"/higher level concurrency
support via the language constructs incorporating the ideas
the Butenhof is talking about (AFAICT) from things like Ada
and perhaps some "research" works (not so widely popular,
"robust", etc) -- uC++:
ftp://plg.uwaterloo.ca/pub/uSystem
and these two papers[1]:
ftp://ftp.dsg.cs.tcd.ie/pub/doc/dsg-86.ps.gz
(Ciaran McHale, Synchronisation in Concurrent,
Object-oriented Languages: Expressive Power,
Genericity and Inheritance, 1994)
http://www.doc.ic.ac.uk/~sjg/thesis-mybg.ps.gz
(Michael Ya'akov Ben-Gershon, An Object-Oriented
Approach to Concurrency and Synchronisation, 2000)
....
[1] Just in case you will have some problems with
the links above (reportedly do NOT always
work), I've got PDFs on my site too:
http://www.terekhov.de/OO-Concurrence/dsg-86.pdf
(McHale's paper)
http://www.terekhov.de/OO-Concurrence/thesis-mybg.pdf
(Ben-Gershon's paper)
Please let me know if that's somehow
"illegal" and I'll just pull these
PDFs from my site away, immediately."
regards,
alexander.
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std...@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]
Alexander Terekhov
Victor Bazarov wrote:
[...]
> > A) Your 'conversations on this subject off-topic'-bullshit:
> >
> >
> http://groups.google.com/groups?selm=1T7H8.24863%24YI5.411928%40twister.tamp
> abay.rr.com
> > (Pay some attention to "Newsgroups: ....")
>
> comp.lang.c++.moderated. Am I supposed to derive some knowledge
> from that fact?
Yes.
> Are you saying that the mere appearance of a
*THE SAME ORIGINAL*
> post on parallel programming in a different
C++ chartered/moderated/'off-topic'-stuff-gets-simply-
bounced>>offline/offband<<---does-*NOT*-show-up-there.
> newsgroup makes it ALRIGHT to post it here? I DON'T THINK SO.
Take some medicine, really. Do something to 'fix'
your 'imagination' and 'logic'-comprehension units.
> > B) Your 'volatile'-bullshit:
> >
> > http://groups.google.com/groups?threadm=a8hgtr%24euj%241%40news.hccnet.nl
> > (Subject: Hardware port)
> >
> >
> http://groups.google.com/groups?selm=L9JR7.478%24BK1.14104%40news.cpqcorp.ne
> t
> > (Subject: Re: keyword volatile required ?)
>
> So, those are links to prevoiusly conducted conversations on hardware
> and threading. Both are irrelevant here, since they have nothing to
> do with the subject at hand: parallel programming has no place in a C++
> newsgroup. Or do you think otherwise? I said that C++ has no mechanisms
> for parallel programming. I said that maybe, may be, 'volatile' and
> 'function reentrability' have some distant relevance, but nothing else.
which is simply confusing and NOT helpful -- aka 'bullshit'.
> You replied with "bullshit". Why?
See above.
> Probably because you're an MVP.
Bullshit.
> I can find no other explanation.
Try again.
> > Well, apart from your mysterious 'language elements' [please
> > explain!] for 'reentrability'... A + B ~= 'total bullshit',
> > or am I wrong?
>
> You _are_ wrong. What the f***
Why not simply write SUKA BL<a couple of '*'> , etc? ;-)
Please be MORE explicit!
> does parallel programming have
> to do with C++ language? Until proven otherwise, it _is_ off-topic
> in comp.lang.c++.
it (i.e. original post) _is_ ON-topic even in comp.lang.c++.moderated,
stupid.
[...]
> > Victor, I love you! You are great!! Is it better now? ;-)
>
> Oh, much! I am all tears... Coming from you, it's so touching...
> NOT!!! Just another attempt at an insult. Failed!
So, why do you then complain/whine to the entire C++ world,
so to speak? ;-) ;-)
> Have a good Saturday.
Thanks. You too, Sunday/Monday/... including!
regards,
alexander.
Alexander Terekhov
Edwin Robert Tisdale wrote:
>
> Scott Robert Ladd wrote:
>
> > Is anyone doing parallel programming in C++?
>
> Well, I guess that I am.
And >>Pentagon<< too (well, 'I guess', and... IRS, etc. aside).
;-) ;-)
regards,
alexander.
Branimir Maksimovic
"Alexander Terekhov" <tere...@web.de> wrote in message
news:3CEF9480...@web.de...
>
> Branimir Maksimovic wrote:
> [...]
> > Well, first try to post to right newsgroups.
>
> Bullshit.
I mean, op will probably get more informations in right news groups.
OP mentioned wide area of subjects (and I don't see how responses
from clc++ can be much relevant to his research on this subjects).
>
> What is the proper newsgroup to discuss, for example,
> the stuff below?
>
> http://www.terekhov.de/thread_ptr.txt
> http://www.terekhov.de/mythread.c
>
comp.programming.threads ? I see in their faq lot of usefull
informations about c++ and threads.
> John Nagle wrote:
> [...]
> > C++ really does need language-level thread
> > support, because there are so many language-level
> > thread issues, of which this is only one. Yes,
> > you can do everything with OS calls, but the
> > language isn't giving you any help in getting it
> > right. Are there any good designs for this being
> > proposed?
>
> FYI:
>
> http://groups.google.com/groups?as_umsgid=3CA9EDE3.2227EE4A%40web.de
>
> "....
> The next step could be that "full"/higher level concurrency
> support via the language constructs incorporating the ideas
> the Butenhof is talking about (AFAICT) from things like Ada
> and perhaps some "research" works (not so widely popular,
> "robust", etc) -- uC++:
I must remind you that this should go to comp.std.c++
as it is group that discuss changes in language.
However, I found following in my copy of D & E of C++:
"
Curiously enough, the initial implementation of C with Classes
contained a feature that is not provided by C++, but is often requested.
One could define a function that would implicitly be called before
every call of every member function (except the constructor) and another
that would be implicitly called every return from every member
function (except the destructor). They were called call and return
functions.
They were used to provide synchronization for the monitor class
in the original task library [Stroustrup, 1980b]:
class monitor: object {
/* ......... */
call() { /* grab lock */ }
return() { /* release lock */ }
/* ........ */
};
These are similar in intent to the CLOS :before and :after methods. Call and
return
functions were removed from language because nobody (but me) used them
and because I seemed to have completely failed to convince people that
call()
and return() had important uses."
My appologies if I have typed something incorrectly,
(but I guess that you'll appritiate my effort :)
Greetings, Bane.
Alexander Terekhov
"fuck off my case" (Victor Bazarov <vAba...@dAnai.com>) wrote:
>
[...I don't want this to be archived...]
But I want. Corrected. And I'll even e-mail you that
google link to this historical demonstration of your
intelligence <= AVEGETABLE
>> *IF* UNNEEDED <<
>
> "Alexander Terekhov" <tere...@web.de> wrote...
> >
> > Victor Bazarov wrote:
> > [...]
> > > does parallel programming have
> > > to do with C++ language? Until proven otherwise, it _is_ off-topic
> > > in comp.lang.c++.
> >
> > it (i.e. original post) _is_ ON-topic even in comp.lang.c++.moderated,
> > stupid.
>
> So, moderators in c.l.c.m showed you the light, did they?
> So, go lick their boots and get the fuck off my case, moron.
Alexander Terekhov
Branimir Maksimovic wrote:
>
> "Alexander Terekhov" <tere...@web.de> wrote in message
> news:3CEF9480...@web.de...
> >
> > Branimir Maksimovic wrote:
> > [...]
> > > Well, first try to post to right newsgroups.
> >
> > Bullshit.
>
> I mean, op will probably get more informations in right news groups.
> OP mentioned wide area of subjects (and I don't see how responses
> from clc++ can be much relevant to his research on this subjects).
Well, >>this response<< (see 'P.S.' below) IS relevant and rather
helpful (I hope) to YOUR research on this subjects... So, the same
could perhaps/probably happen w.r.t. OP as well, I guess. ;-)
> > What is the proper newsgroup to discuss, for example,
> > the stuff below?
> >
> > http://www.terekhov.de/thread_ptr.txt
> > http://www.terekhov.de/mythread.c
> >
>
> comp.programming.threads ? I see in their faq lot of usefull
> informations about c++ and threads.
Yep. Thanks ('correct' answer is actually *both c.p.t AND c.l.c++*,
comp.parallel and alike aside). ;-)
Hmmm. Well, only because of 'Stroustrup'... NO COMMENTS. ;-)
> class monitor: object {
> /* ......... */
> call() { /* grab lock */ }
> return() { /* release lock */ }
> /* ........ */
> };
>
> These are similar in intent to the CLOS :before and :after methods. Call and
> return
> functions were removed from language because nobody (but me) used them
> and because I seemed to have completely failed to convince people that
> call()
> and return() had important uses."
>
> My appologies if I have typed something incorrectly,
> (but I guess that you'll appritiate my effort :)
No problems, I do appreciate your effort, indeed. Thanks.
regards,
alexander.
P.S. And the example of REAL C++ 'monitor' (well, close
to 'REAL': there ARE a couple of silly things in it,
but NOT 'deadly critical', though) can be found here:
http://www.boost.org/libs/thread/doc/condition.html#examples
(see 'class bounded_buffer')