can perl do multi-threaded programming?

[David M Rosner]

i have a program where i want to kick off several other programs all at
once. i could fork each one into its own process and then exec it, but i'd
rather have the advanatages of true-multithreading and do each one as a
seperate task. i've searched around and i can't seem to find any
multi-threading information on Perl. is there such a thing?

thanks!

- dmr

[Jeremy D. Zawodny]

[original author automagically cc'd via e-mail]

Perl5.005 (currently under development) will have the beginnings of
true multi-threading support.

Jeremy
--
Jeremy D. Zawodny jza...@wcnet.org
Web Server Administrator w...@wcnet.org
Wood County Free Net (Ohio) http://www.wcnet.org/

[Zenin]

David M Rosner <da...@digidem.com> wrote:
: i have a program where i want to kick off several other programs all at
: once. i could fork each one into its own process and then exec it, but i'd
: rather have the advanatages of true-multithreading and do each one as a
: seperate task. i've searched around and i can't seem to find any
: multi-threading information on Perl. is there such a thing?

5.005 will have threads, but it's not ready for prime time yet.
Check the Perl 5 porters mailing list for details:

http://www.rosat.mpe-garching.mpg.de/mailing-lists/perl-porters/

--
-Zenin
ze...@best.com

[Tom Christiansen]

[courtesy cc of this posting sent to cited author via email]

In comp.lang.perl.misc,
"David M Rosner" <da...@digidem.com> writes:
:i have a program where i want to kick off several other programs all at

:once. i could fork each one into its own process and then exec it, but i'd
:rather have the advanatages of true-multithreading and do each one as a
:seperate task. i've searched around and i can't seem to find any
:multi-threading information on Perl. is there such a thing?

Fork is good for you. Here's a demo that shows how powerful
fork is for multithreading:

# make socket SOCFH, connect, then dup the PC
if (fork) { while (<SOCFH>) { print STDOUT $_ } }
else { while (<STDIN>) { print SOCFH $_ } }

All you get from the "threading" is shared data, which is its own
problem. We've done multitasking for years you know with fork.
For most people, non-PC-duplicating threading is a lame excuse
for not understanding the select system call.

Now, it probably is not so in your case, but I advise you
not to jump on the "threads are God" bandwagon too quickly.

--tom
--
Tom Christiansen tch...@jhereg.perl.com
Your csh still thinks true is false. Write to your vendor today and tell
them that next year Configure ought to "rm /bin/csh" unless they fix their
blasted shell. :-) --Larry Wall in Configure from the perl distribution

[Kerry Schwab]

In article <661gsj$3ji$1...@csnews.cs.colorado.edu>,

Tom Christiansen <tch...@mox.perl.com> wrote:
> [courtesy cc of this posting sent to cited author via email]
>
>In comp.lang.perl.misc,
> "David M Rosner" <da...@digidem.com> writes:
>:i have a program where i want to kick off several other programs all at
>:once. i could fork each one into its own process and then exec it, but i'd
>:rather have the advanatages of true-multithreading and do each one as a
>:seperate task. i've searched around and i can't seem to find any
>:multi-threading information on Perl. is there such a thing?
>
>Fork is good for you. Here's a demo that shows how powerful
>fork is for multithreading:
>
> # make socket SOCFH, connect, then dup the PC
> if (fork) { while (<SOCFH>) { print STDOUT $_ } }
> else { while (<STDIN>) { print SOCFH $_ } }
>
>All you get from the "threading" is shared data, which is its own
>problem. We've done multitasking for years you know with fork.
>For most people, non-PC-duplicating threading is a lame excuse
>for not understanding the select system call.
>

I suppose this is true at a very basic level, but cases where
multi-threaded perl might be nicer than fork:

1) NT,95, other platforms where fork() isn't available.

2) X11 modules (TkPerl?) where forked children can't muck with
windows created by the parent. Don't you hate that hourglass ?
(And yes, I've tried processing bits at the top of the event
loop, but you end up freezing the gui lots)

3) Complex multi socket connections where native multithreading
provides better timeslicing (esp while reading and writing)
than one can hope for using select()). I guess a good example would
be some sort of interactive chat server. I have no doubt that
Tom could do this without multi-threading, but I also suppose
it would be less complex using threads.

Having been involved with user-space pthreads before, I would concede
that user-space threads are (almost) useless.

--
Kerry

[Gary Howland]

Tom Christiansen wrote:
>
> [courtesy cc of this posting sent to cited author via email]
>
> In comp.lang.perl.misc,
> "David M Rosner" <da...@digidem.com> writes:
> :i have a program where i want to kick off several other programs all at
> :once. i could fork each one into its own process and then exec it, but i'd
> :rather have the advanatages of true-multithreading and do each one as a
> :seperate task. i've searched around and i can't seem to find any
> :multi-threading information on Perl. is there such a thing?
>
> Fork is good for you. Here's a demo that shows how powerful
> fork is for multithreading:
>
> # make socket SOCFH, connect, then dup the PC
> if (fork) { while (<SOCFH>) { print STDOUT $_ } }
> else { while (<STDIN>) { print SOCFH $_ } }

Tom's at it again, with that little program of his. This is not
multithreading, it's multiprocessing. fork() is a pathetic solution.

> All you get from the "threading" is shared data, which is its own

^^^!
> problem.

It solves a damn sight more problems than it creates. Anyway,
multithreading gives you more than shared data - it gives you the
ability to stop what you're doing and resume it at a later time, without
the hassle of having to save and restore the program state.

> We've done multitasking for years you know with fork.

And people write applications in COBOL, but it doesn't mean it's the
best solution. You don't see programs like netscape using fork, do
you? Imagine if it kicked off a process for every thread!!!!

> For most people, non-PC-duplicating threading is a lame excuse
> for not understanding the select system call.

The select call does not solve the problem of being able to save and
restore your state, but threads do. If I use select, then every time I
want to do some I/O, I must save my state, return back to the select
loop, read the next character (or whatever), and restore myself when
selected again!!! Select() is a pathetic "solution". (Unless of course
we have a save-state/restore-state facility (coroutines), in which case
select is a fine solution.

In fact, we can easily implement threads in perl using select, if only
we had a pair of perl instructions ("suspend" and "resume"?), that save
and restore the program state. This facility would be damned useful for
all sorts of other things too, such as backtracking. But it's not going
to happen, since it's too useful, too (java/python/lisp)-like, and can
implemented in perl as it currently stands (turing machine equivalency
and all that). Like threads.

Gary
--
pub 1024/C001D00D 1996/01/22 Gary Howland <ga...@hotlava.com>
Key fingerprint = 0C FB 60 61 4D 3B 24 7D 1C 89 1D BE 1F EE 09 06

[Gary Howland]

Kerry Schwab wrote:
>
> Having been involved with user-space pthreads before, I would concede
> that user-space threads are (almost) useless.

Of course our experience depends upon many factors (such as the project
being developed, the language being used, etc.), but my personal
experience has been just the opposite - that user-space threads work
exceedingly well, and even have certain advantages over kernel threads.

[Tom Christiansen]

[courtesy cc of this posting sent to cited author via email]

In comp.lang.perl.misc,
Gary Howland <ghow...@hotlava.com> writes:
:Tom Christiansen wrote:
:Tom's at it again, with that little program of his. This is not

:multithreading, it's multiprocessing. fork() is a pathetic solution.

You're playing word games. The code is clearly multitasking: there are
two PCs running concurrently. And I can't see what you mean by pathetic.
You trying to pick a fight? That's two messages in a row I've read from
you like this. I'd really rather not.

--tom
--
Tom Christiansen tch...@jhereg.perl.com

Does the same as the system call of that name.
If you don't know what it does, don't worry about it.
--Larry Wall in the perl man page regarding chroot(2)

[Eryq]

On a slightly-related issue regarding 5.005:

Up until now, I've worked with the assumption that signal
handlers in Perl must follow the traditions of their C cousins:
avoid non-reentrant malloc and stdio libraries
at all costs... so, for example, your Perl sighandler *can* safely use
syswrite() for output, and it *can* set a pre-declared scalar variable
to an integer or a float (or a string which will fit in the
scalar's current buffer), and it *can* call subroutines which
do only these things... but little else.

Now, sfio (I believe) allows for reentrancy, and it seems that
any malloc used by a thread-safe Perl would have to be reentrant
as well. So... does this mean that signal handlers under 5.005
will no longer have to walk on eggshells to avoid core dumps?

--
___ _ _ _ _ ___ _ Eryq (er...@zeegee.com)
/ _ \| '_| | | |/ _ ' / President, Zero G Inc.
| __/| | | |_| | |_| | "Talk is cheap. Let's build." - Red Green.
\___||_| \__, |\__, |___/\ Visit STREETWISE, Chicago's newspaper by/
|___/ |______/ of the homeless: http://www.streetwise.org

[Malcolm Beattie]

In article <3486E9...@zeegee.com>, Eryq <er...@zeegee.com> wrote:
>On a slightly-related issue regarding 5.005:
>
>Up until now, I've worked with the assumption that signal
>handlers in Perl must follow the traditions of their C cousins:
>avoid non-reentrant malloc and stdio libraries
>at all costs... so, for example, your Perl sighandler *can* safely use
>syswrite() for output, and it *can* set a pre-declared scalar variable
>to an integer or a float (or a string which will fit in the
>scalar's current buffer), and it *can* call subroutines which
>do only these things... but little else.
>
>Now, sfio (I believe) allows for reentrancy, and it seems that
>any malloc used by a thread-safe Perl would have to be reentrant
>as well. So... does this mean that signal handlers under 5.005
>will no longer have to walk on eggshells to avoid core dumps?

There are two ways to allow arbitrary perl code to be usable from
signal handlers. One is Chip's way which has the advantage that a
multi-threading perl isn't required but has the disadvantage that
perl's internal loop has to poll after every few instructions
whether a signal has been raised or not. The other is the way that
I've included in the threading stuff: it has the disadvantage that
a multi-threaded perl is required but the advantage that no polling
is necessary in perl's internals. The multi-threaded safe signal
handling works by creating a special thread which sits there
blocking waiting to be notified that a signal has arrived. The
real C signal handler merely notifies the signal thread (safely)
about the arrival of the signal. The signal handling thread then
executes whatever perl code you like. So if you include
use Thread::Signal;
at the top of your program (and wait until I've polished up the
internals a bit to complete things) then you can write signal
handlers that do whatever you like completely safely and without
affecting the performance of your main program. Note, however,
that a perl compiled with multi-threading support
(./Configure -Dusethreads) is intrinsically slower than perl
compiled without because of the extra dereferencing necessary.

--Malcolm

--
Malcolm Beattie <mbea...@sable.ox.ac.uk>
Oxford University Computing Services
"I permitted that as a demonstration of futility" --Grey Roger

[Ilya Zakharevich]

In article <6692m4$6b7$1...@news.ox.ac.uk>,

Malcolm Beattie <mbea...@sable.ox.ac.uk> wrote:
> There are two ways to allow arbitrary perl code to be usable from
> signal handlers. One is Chip's way which has the advantage that a
> multi-threading perl isn't required but has the disadvantage that
> perl's internal loop has to poll after every few instructions
> whether a signal has been raised or not.

The other disadvantage is that it does not work. Not in the sense
that the current signal handlers do not work (they may do whatever
they are pleased to do, most probabably segfault). But in the sense
that they cannot stop whatever is going now, they are postponed until
next sequence point, which may happen the next millenium if some nasty
internal perl operation take a millenium.

Anyway, your objection is not that important, since the common wisdom
is that it is enough to check for "delayed" signals from OP_UNSTACK
opcode.

> The other is the way that
> I've included in the threading stuff: it has the disadvantage that
> a multi-threaded perl is required but the advantage that no polling
> is necessary in perl's internals. The multi-threaded safe signal
> handling works by creating a special thread which sits there
> blocking waiting to be notified that a signal has arrived. The
> real C signal handler merely notifies the signal thread (safely)
> about the arrival of the signal. The signal handling thread then
> executes whatever perl code you like. So if you include
> use Thread::Signal;
> at the top of your program (and wait until I've polished up the
> internals a bit to complete things) then you can write signal
> handlers that do whatever you like completely safely and without
> affecting the performance of your main program. Note, however,
> that a perl compiled with multi-threading support
> (./Configure -Dusethreads) is intrinsically slower than perl
> compiled without because of the extra dereferencing necessary.

But you also cannot abort a long elementary operation this way too,
since you cannot longjump the other thread.

Most promising way is to have all this mechanisms (Chips, threaded,
and as a last resort the unsafe current mechanism) available to the
users discretion. Like this:

$SIG{INT} = {delayed => \&myhandler1, immediate => \&myhandler2};

with the semantic that the service of SIGINT is delayed until the next
OP_UNSTACK, if the second one arrives before the first one is
serviced, then it is executed immediately.

Ilya

[Sriram Srinivasan]

>
> The select call does not solve the problem of being able to save and
> restore your state, but threads do. If I use select, then every time
> I
> want to do some I/O, I must save my state, return back to the select
> loop, read the next character (or whatever), and restore myself when
> selected again!!! Select() is a pathetic "solution". (Unless of
> course
> we have a save-state/restore-state facility (coroutines), in which
> case
> select is a fine solution.

Having programmed the last six months or so in java, I find myself
missing select terribly. The JDK unfortunately doesn't have
select, so one is forced to either allocate a thread to listen on
every socket or poll every socket, both of which are fairly painful
solutions.

To get back to the other argument (fork vs. threads), I prefer
multiple threads over multiple processes, unless I absolutely want
a different protected address space. The only time I have needed this
in recent years is while doing transaction coordination, which
one doesn't want clobbered by bugs in one's code.

OTOH, I have not worked on machines like Tandem, where processes
map well to processors, inter-process messaging (using pathsend)
is exceedingly fast and where threads don't take advantage of the
hardwre.

________________________________________________________________________
Principal Engineer WebLogic, San Francisco www.weblogic.com
"Advanced Perl Programming" : http://www.ora.com/catalog/advperl/

[Gary Howland]

Sriram Srinivasan wrote:
>
> >
> > The select call does not solve the problem of being able to save and
> > restore your state, but threads do. If I use select, then every time
> > I
> > want to do some I/O, I must save my state, return back to the select
> > loop, read the next character (or whatever), and restore myself when
> > selected again!!! Select() is a pathetic "solution". (Unless of
> > course
> > we have a save-state/restore-state facility (coroutines), in which
> > case
> > select is a fine solution.
>
>
> Having programmed the last six months or so in java, I find myself
> missing select terribly. The JDK unfortunately doesn't have
> select, so one is forced to either allocate a thread to listen on
> every socket or poll every socket, both of which are fairly painful
> solutions.

I'm a bit puzzled as to why you find it painful allocating a thread for
every socket - surely dealing with an array of Threads isn't much
different to dealing with an array of file descriptors? I suppose if
you're writing something like inetd, you may be a little uncomfortable
starting up a whole bunch of threads. This may be especially true if
you've previously been used to using fork(), and are perhaps expecting
Threads to be relatively "expensive" - however, a program should be able
to run with hundreds, if not thousands, of threads without too much
trouble (I don't know what the Perl overhead per thread is going to be,
though).

Still, it's a shame java doesn't provide access to select() though. (But
I'd be happy if that were the worst of it's problems)

[Sriram Srinivasan]

Gary Howland wrote:
>
> I'm a bit puzzled as to why you find it painful allocating a thread
> for
> every socket - surely dealing with an array of Threads isn't much
> different to dealing with an array of file descriptors? I suppose if
> you're writing something like inetd, you may be a little uncomfortable
> starting up a whole bunch of threads. This may be especially true if
> you've previously been used to using fork(), and are perhaps expecting
> Threads to be relatively "expensive" - however, a program should be
> able
> to run with hundreds, if not thousands, of threads without too much
> trouble (I don't know what the Perl overhead per thread is going to
> be,
> though).

Oh, it is not an ease of programming issue; in fact threads are
make it considerably easier. But they are terribly expensive in terms
of resources. With 1000 clients and a thread per client (socket),
and assuming a default stack size of 128k, you are talking 128M
of just stack. Then there is the additional burden on the scheduler
for managing these numbers of threads. Contrast this with the
amount of effort it takes to set the appropriate bit in a bitmap,
and then wake up the select. Sure you have to spend additional
cycles poking inside the bitmap to see what is available, but
surely that is done really fast.

Threads don't scale well beyond the few hundreds, on conventional
hardware. But coming back to the original topic, I find
threads to be eminently more useful than processes, for practically
all my tasks.

> Still, it's a shame java doesn't provide access to select() though.
> (But I'd be happy if that were the worst of it's problems)

Microsoft's JDK does, actually, and we use it where we can.

- Sriram

[Jacqui Caren]

In article <666l7h$e06$2...@csnews.cs.colorado.edu>,

Tom Christiansen <tch...@mox.perl.com> wrote:
> [courtesy cc of this posting sent to cited author via email]
>
>In comp.lang.perl.misc,
> Gary Howland <ghow...@hotlava.com> writes:
>:Tom Christiansen wrote:
>:Tom's at it again, with that little program of his. This is not
>:multithreading, it's multiprocessing. fork() is a pathetic solution.
>
>You're playing word games. The code is clearly multitasking: there are
>two PCs running concurrently. And I can't see what you mean by pathetic.
>You trying to pick a fight? That's two messages in a row I've read from
>you like this. I'd really rather not.

Tom,
If I had posted your example, I would have (quite rightly) been shot
down in flames by anyone who understands the difference between process
and thread models. I don't want to start a war about what is a process
and what is a thread - I don't even want to start talking about physical
and logical thread models ala Cray's approach. For many in clp.misc who
are probably CW's, your posting could be quite easily misinterprested to
mean fork() is how you multithread a program.

I and many others here look to people such as yourself (and the rest of
the p5p team) to provide concise and accurate responses, Your response
in this case was well below your normal, very high standards.

Please do not take offence at the above. Your contributions are welcomed
by lesser perl people such as myself :-), and I hope that this is not
seen as any form of personal attack.

All the best,
Jacqui
--
Jacqui Caren Email: Jacqui...@ig.co.uk
Paul Ingram Group Fax: +44 1483 419 419
140A High Street Phone: +44 1483 424 424
Godalming GU7 1AB United Kingdom