Dividing a problem into subproblems and using mulitiple CPU's and perl in linux

[mathem...@gmail.com]

Dear Sir/Madame,

I have a couple of questions regarding multi-tasking using perl in a
unix multi-CPU environment.

I have a unix system with 4 CPU's. If i can divide the solution of a
problem into 4 independent subproblems, would there be a way then to
run 4 perl programs, each assigned one of the 4 problems for
processing? Each should use one of the four CPU's on the same machine
and somehow report to a main program upon finishing, so that the
partial results can be "merged".

As you see, it is more of a parallel programming question using perl.
Is the answer related to "forking child processes" ? Is forking child
processes and giving each child a chunk would be the best solution? Or
am I wrong? I am not very familiar with the unix and perl.

Do you have any information about a good tutorial about multi-tasking
with perl?

Thanks a lot for your response.

M.

[Peter Makholm]

mathem...@gmail.com writes:

> processing? Each should use one of the four CPU's on the same machine
> and somehow report to a main program upon finishing, so that the
> partial results can be "merged".

The right solutions depends on how you need to communicate between the
master process and the children. If you don't need any communication
during the processing of a job and you return value is a simple
success/failure then I would prefer to use something simple like
Parallel::ForkManager:

my $pm = new Parallel::ForkManager(8);

$pm->run_on_finish( sub {
my ($pid, $exit_code, $ident) = @_;

print $job->id() . " ended with " .
( $exit_code >> 8 ? "success" : "failure");
} );

while( defined( $job = shift @jobqueue ) ) {
$pm->start($job) and next;

$result = process($job);

$pm->finish($result);
}

$pm->wait_all_children;

> As you see, it is more of a parallel programming question using perl.
> Is the answer related to "forking child processes" ? Is forking child
> processes and giving each child a chunk would be the best solution? Or
> am I wrong? I am not very familiar with the unix and perl.

The above is the "forking child" solution. Wether this is the right
solution depends on you problem. If you need a more complicated
feedback the "forking child" solution becomes more difficult. A
solution would be passing the child a filehandle and write the result
to this. You run_on_finish hook would the read back the result.

But I don't think this will ever be true parallel programming.

//Makholm

[mathem...@gmail.com]

On May 26, 3:48 pm, Peter Makholm <pe...@makholm.net> wrote:

Thanks a lot Makholm.