Synchronized / Thread syntax in Perl 6

[John Drago]

I asked this via the Google Groups interface a few weeks ago, but I'm not sure if it made it here.
I am asking again in case the question never made it onto the list.

Has the syntax for synchronized/threaded @things been worked out?
For example:

class Foo is synchronized {
...
}

our method Bar is synchronized {
...
}

class Baz {
has $.Bux is synchronized;
}

...or is there some new, less Java-esque way to express "only one thread may access this thing at a time"?

________________________________________
John Drago | VP Software Engineering
john....@precissystems.com
www.precissystems.com

[Juerd]

James Mastros skribis 2006-05-31 12:03 (+0100):
> I don't like the name synchronized -- it implies that multiple things are
> happening at the same time, as in synchronized swiming, which is exactly the
> opposite of what should be implied. "Serialized" would be a nice name,
> except it implies serializing to a serial format, like disk. "Locked" is
> the best name I can think of, and it frankly isn't that good -- it's so
> vauge as to be able to mean almost anything.

is exclusive

Juerd
--
http://convolution.nl/maak_juerd_blij.html
http://convolution.nl/make_juerd_happy.html
http://convolution.nl/gajigu_juerd_n.html

[James Mastros]

On Tue, May 30, 2006 at 03:41:06PM -0600, John Drago wrote:
> I asked this via the Google Groups interface a few weeks ago, but I'm not sure if it made it here.
> I am asking again in case the question never made it onto the list.
>
> Has the syntax for synchronized/threaded @things been worked out?
> For example:
>
> class Foo is synchronized {
> ...
> }

I don't like the name synchronized -- it implies that multiple things are

happening at the same time, as in synchronized swiming, which is exactly the
opposite of what should be implied. "Serialized" would be a nice name,
except it implies serializing to a serial format, like disk. "Locked" is
the best name I can think of, and it frankly isn't that good -- it's so
vauge as to be able to mean almost anything.

(Also, of course, all those /z/ names should have a s/z/s/ version, for
those who speak a z-impared dialect of English.)

-=- James Mastros

[Sage La Torra]

We could always go with the Windows API "Critical Section" name. Locked is
probably as good a descriptor, and avoids anything associated with Windows.

Sage

[Paul Hodges]

How about one of these?
======================

class Baz {
has $.a is restricted;
has $.b is controlled;
has $.c is unique;
has $.d is shared;
has $.e is queued;
has $.f is token;
...
}

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

[Benjamin Smith]

On Tue, May 30, 2006 at 03:41:06PM -0600, John Drago wrote:

> class Foo is synchronized {
> ...
> }
>
> our method Bar is synchronized {
> ...
> }
>
> class Baz {
> has $.Bux is synchronized;
> }

To everyone participating in this thread:
There has already been a draft spec for concurrency written, please see
http://svn.perl.org/perl6/pugs/trunk/docs/Perl6/Spec/Concurrency.pod .

It suggests "is critical;" for the first, and second, but I can't see
anything like the third. It also suggests that Perl6 will have
transactions and so will also have "is atomic".

(There's a Google Summer of Code project to add software transactional
memory primitives to parrot.)

--
Benjamin Smith <bsm...@vtrl.co.uk, benjami...@yahoo.co.uk>

[John Drago]

Thanks to all who have read or replied -

I'm reading the Concurrency POD right now - more questions when I'm done.

________________________________

John Drago | VP Software Engineering

john....@precissystems.com
www.precissystems.com

[John Drago]

James Mastros wrote:

> I don't like the name synchronized -- it implies that multiple things are
> happening at the same time, as in synchronized swiming, which is exactly the
> opposite of what should be implied. "Serialized" would be a nice name,
> except it implies serializing to a serial format, like disk. "Locked" is
> the best name I can think of, and it frankly isn't that good -- it's so
> vauge as to be able to mean almost anything.
>
> (Also, of course, all those /z/ names should have a s/z/s/ version, for
> those who speak a z-impared dialect of English.)
>
> -=- James Mastros
>

Agreed - maybe "is serial" instead, which suggests "is parallel" would be its implicit counterpart.

If we have a situation that looks like this:

our method TakesForever ( int $num is serial ) is async {
# Do something that takes a long time...then:
$num++;
# $num has not been changed by anything else that might
# have access to $num.
}

my $age = 27;
TakesForever( $age );
$age += 20; # Fails somehow, because &TakesForever is working on $num

Maybe a better example is needed - this one is pretty contrived.

[Paul Hodges]

--- John Drago <john....@precissystems.com> wrote:

> James Mastros wrote:
> > I don't like the name synchronized -- it implies that multiple
> > things are happening at the same time, as in synchronized swiming,
> > which is exactly the opposite of what should be implied.
> > "Serialized" would be a nice name, except it implies serializing
> > to a serial format, like disk. "Locked" is the best name I can
> > think of, and it frankly isn't that good -- it's so vauge as to
> > be able to mean almost anything.

> > . . .

>
> Agreed - maybe "is serial" instead, which suggests "is parallel"
> would be its implicit counterpart.

You mean "is parallel" as a synonym for "is async"? I actually like it
better, but that's just me.

> If we have a situation that looks like this:
>
> our method TakesForever ( int $num is serial ) is async {
> # Do something that takes a long time...then:
> $num++;
> # $num has not been changed by anything else that might
> # have access to $num.
> }
>
> my $age = 27;
> TakesForever( $age );
> $age += 20; # Fails somehow, because &TakesForever is working on $num

Hmm....
Is "fails somehow" the default?
I was thinking the better default would be more like standard
threading.
If $age has been passed to an asynchronous closure, it should be marked
as locked, and other threads trying to access it would have to get a
lock first. Yes, lots of overhead.... but that way if the default is
WAIT (which seems the smart default to me), the thread waits until
TakesForever() releases the resource.

if we declare

our method TakesForever ( int $num is serial ) is async but NOWAIT {
...
}

or

my $age = 27 but NOWAIT;

or

TakesForever( $age but NOWAIT );

(or whatever) then I'd say it should just fail. I mean, isn't that kind
of the idea, to have that sort of flexibility?

> Maybe a better example is needed - this one is pretty contrived.

I dunno. It gets a point across.

[John Drago]

> > James Mastros wrote:
> > > I don't like the name synchronized -- it implies that multiple
> > > things are happening at the same time, as in synchronized swiming,
> > > which is exactly the opposite of what should be implied.
> > > "Serialized" would be a nice name, except it implies serializing
> > > to a serial format, like disk. "Locked" is the best name I can
> > > think of, and it frankly isn't that good -- it's so vauge as to
> > > be able to mean almost anything.
> > > . . .
> >
> > Agreed - maybe "is serial" instead, which suggests "is parallel"
> > would be its implicit counterpart.
>
> You mean "is parallel" as a synonym for "is async"? I actually like it
> better, but that's just me.

I think "is parallel" denotes something as usable by multiple threads simultaneously, "in parallel".
"is serial" would denote that only one thread can use the $thing at a time, exclusively.

>
> > If we have a situation that looks like this:
> >
> > our method TakesForever ( int $num is serial ) is async {
> > # Do something that takes a long time...then:
> > $num++;
> > # $num has not been changed by anything else that might
> > # have access to $num.
> > }
> >
> > my $age = 27;
> > TakesForever( $age );
> > $age += 20; # Fails somehow, because &TakesForever is working on $num
>
> Hmm....
> Is "fails somehow" the default?

After reading your comment, I realize the whole direction I started thinking about this is all wrong.
More on that in a minute, after coffee.

> I was thinking the better default would be more like standard
> threading.
> If $age has been passed to an asynchronous closure, it should be marked
> as locked, and other threads trying to access it would have to get a
> lock first. Yes, lots of overhead.... but that way if the default is
> WAIT (which seems the smart default to me), the thread waits until
> TakesForever() releases the resource.
>
> if we declare
>
> our method TakesForever ( int $num is serial ) is async but NOWAIT {
> ...
> }
>
> or
>
> my $age = 27 but NOWAIT;
>
> or
>
> TakesForever( $age but NOWAIT );
>
> (or whatever) then I'd say it should just fail. I mean, isn't that kind
> of the idea, to have that sort of flexibility?
>

Perhaps some more syntax-play is in order here.

<disclamer>
I should note that my only experience with threading under Perl5 is via threads.pm (and forks.pm).
I have not written multi-threaded applications using anything else (Java, .Net, etc).
I have, however written several critical (to our business) applications that depend on threads.pm and/or forks.pm
</disclamer>

One thing about threading with Perl5 is that it is easy to write a simple threaded program that is entirely opaque - unless you
wrote the program yourself.

The program below gets a list of coderefs and executes each one, then returns the result.
#================================================
(Using the threads.pm way...)
package QueueRunner;

use strict;
use threads;
use threads::shared;

sub process_job_queue
{
my ($s, @jobs_in) = @_;
my @results : shared = ();
my @workers = ();

push @workers, async { push( @results, $_->() ) } foreach @jobs_in;
$_->join foreach @workers;
return @results;
}# end process_job_queue()

# Elsewhere...
package main;

my @answer = QueueRunner->process_job_queue( \&job1, \&job2, \&job3 );

#================================================

And my attempt at the same, using Perl6:
#================================================

class QueueRunner {
our sub process_job_queue( Code @jobs_in ) returns List of Any {
my Any @results is parallel;
my Thread @workers = ();

for @jobs_in {
@workers.push( async { &_() } );
}
for @workers {
@results.push( $_.join() );
}

return @results;
}# end process_job_queue()
}# end QueueRunner

# Elsewhere...
my @answer = QueueRunner.process_job_queue( @jobs );

#================================================

I made absolutely no progress here. It seems to me that it's no more obvious what's going on here than in the Perl5 version. Any
comments?

Regards,
John Drago

[Paul Hodges]

--- John Drago <john....@precissystems.com> wrote:

> > You mean "is parallel" as a synonym for "is async"?
>

> I think "is parallel" denotes something as usable by multiple threads
> simultaneously, "in parallel".
> "is serial" would denote that only one thread can use the $thing at a
> time, exclusively.

Are you saying both are asynchronous, but one specifies that a resource
should be locked rather than duplicated?

> . . .

> > If $age has been passed to an asynchronous closure, it should be
> > marked as locked, and other threads trying to access it would have
> > to get a lock first. Yes, lots of overhead.... but that way if the
> > default is WAIT (which seems the smart default to me), the thread
> > waits until TakesForever() releases the resource.

In that light, responding to my own comment, you could infer that you
should lock anything passed by reference, and not worry about it if
passed by value unless you said "$age is locked" or "is serial" or
whatever, yes?

> > if we declare
> >
> > our method TakesForever ( int $num is serial ) is async but NOWAIT
> {
> > ...
> > }
> >
> > or
> >
> > my $age = 27 but NOWAIT;
> >
> > or
> >
> > TakesForever( $age but NOWAIT );
> >
> > (or whatever) then I'd say it should just fail. I mean, isn't that
> kind
> > of the idea, to have that sort of flexibility?
> >
>
> Perhaps some more syntax-play is in order here.

lol -- yeah. :)

Hm. If we're using *implicit* threads (which I thought was the point),
how about

class QueueRunner {

our sub process_queue(Code @jobs_in) {
my @ans is serial;
@ans.push map { async { &_() } } @jobs_in;

@ans;
}

}# end QueueRunner

# Elsewhere...
my @answer = QueueRunner.process_job_queue( @jobs );

The point is that the call to async() is supposed to handle the thread
management for you, isn't it?

Though if that works, you could squish this example even more, to

class QueueRunner {

our sub process_queue(Code @jobs_in) {
map { async { &_() } } @jobs_in;
}

}# end QueueRunner

# Elsewhere...
my @answer = QueueRunner.process_job_queue( @jobs );

and the issues of serialization are hidden in the map() call. For all
that....

my @answer = map { async { &_() } } @jobs;

though that gets away from the point.

Someone smack me if I'm drifting too far here?

[John Drago]

> > > You mean "is parallel" as a synonym for "is async"?
> >
> > I think "is parallel" denotes something as usable by multiple threads
> > simultaneously, "in parallel".
> > "is serial" would denote that only one thread can use the $thing at a
> > time, exclusively.
>
> Are you saying both are asynchronous, but one specifies that a resource
> should be locked rather than duplicated?
>
> > . . .
> > > If $age has been passed to an asynchronous closure, it should be
> > > marked as locked, and other threads trying to access it would have
> > > to get a lock first. Yes, lots of overhead.... but that way if the
> > > default is WAIT (which seems the smart default to me), the thread
> > > waits until TakesForever() releases the resource.
>
> In that light, responding to my own comment, you could infer that you
> should lock anything passed by reference, and not worry about it if
> passed by value unless you said "$age is locked" or "is serial" or
> whatever, yes?

Correct - since only references can affect their originator (or whatever you call it).
Modifying something passed by value would be exempt.

Actually I think you did it just right. I think that horse is dead now.

So, what about "serial" classes and methods?:

# Does marking this class "is serial" mean it's forced to be a singleton?
class Foo is serial {
has $.counted = 0;
our method Qux {
say($.counted++ ~ " Foo.Qux");
}
}

# Only one thread may call Bar.Baz() at a time:
class Bar {
has $.counted = 0;
our method Baz is serial {
say($.counted++ ~ " Bar.Baz");
}
}

# ... later ...

my Foo $foo = Foo.new();
my Bar $bar = Bar.new();

# $foo and $bar are implicitly shared with other threads:
for 1..5 {
async {
is atomic;
$foo.Qux();
$bar.Baz();
};
}

...what would it print?
1 Foo.Qux
1 Bar.Baz
2 Bar.Baz
2 Foo.Qux
3 Bar.Baz
4 Bar.Baz
3 Foo.Qux
5 Bar.Baz
6 Bar.Baz
4 Foo.Qux
7 Bar.Baz
8 Bar.Baz
5 Foo.Qux
9 Bar.Baz
10 Bar.Baz
(Now the first thread would be done with $foo, so the second thread could use it.)
6 Foo.Qux
7 Foo.Qux
8 Foo.Qux
9 Foo.Qux
10 Foo.Qux

-----------------------------------------------
Regards,
John Drago

[Paul Hodges]

--- John Drago <john....@precissystems.com> wrote:

. . .

> > class QueueRunner {
> > our sub process_queue(Code @jobs_in) {
> > my @ans is serial;
> > @ans.push map { async { &_() } } @jobs_in;
> > @ans;
> > }
> > }

> > my @answer = QueueRunner.process_job_queue( @jobs );
>

> Actually I think you did it just right.
> I think that horse is dead now.

LOL!! I'm flattered. =o)

> So, what about "serial" classes and methods?:
>
> # Does marking this class "is serial" mean it's forced to be a
> singleton?

Hmm.... I wouldn't think so. You should still be able to spawn object
instances, but I'd say it scopes the lock to the whole class, so that
everything in that container is locked any time anything in the class
in accessed. It's a cheap way to make all shared resources queue up
nicely, if you just want a quick and dirty script instead of something
well streamlined. Kind of like locking at the DB or table level instead
of just the row, but there are times when that's what you need.

. . .

[Ashley Winters]

On 6/2/06, Paul Hodges <ydb...@yahoo.com> wrote:
> Though if that works, you could squish this example even more, to
>
> class QueueRunner {
>
> our sub process_queue(Code @jobs_in) {
> map { async { &_() } } @jobs_in;
> }
>
> }# end QueueRunner
>
> # Elsewhere...
> my @answer = QueueRunner.process_job_queue( @jobs );
>
> and the issues of serialization are hidden in the map() call. For all
> that....
>
> my @answer = map { async { &_() } } @jobs;
>
> though that gets away from the point.
>
> Someone smack me if I'm drifting too far here?

That still seems too explicit. I thought we had hyperoperators to
implictly parallelize for us:

my @answer = @jobs.»();

Which would run them in parallel automatically, if possible.

- Ashley Winters

[Paul Hodges]

--- Ashley Winters <ashley....@gmail.com> wrote:
> On 6/2/06, Paul Hodges <ydb...@yahoo.com> wrote:
> >
> > my @answer = map { async { &_() } } @jobs;
>

> That still seems too explicit. I thought we had hyperoperators to
> implictly parallelize for us:
>
> my @answer = @jobs.»();
>
> Which would run them in parallel automatically, if possible.

Snazzy bit of syntactic shenanigans, that...and slick, if we're in that
mode, but just to clarify, I *will* still be able to know whether or
not something's going to thread? Or will it matter?

Seems to me there will be times when I WANT a purely single-threaded
system, even if I'm using hyperops -- in fact, especially if I'm using
hyperops. Maybe we need a pragma to be able to step in and out as
needed?

{ no threads;
print @_.»();

[Larry Wall]

On Sat, Jun 03, 2006 at 03:51:45PM -0700, Paul Hodges wrote:
: --- Ashley Winters <ashley....@gmail.com> wrote:
: > On 6/2/06, Paul Hodges <ydb...@yahoo.com> wrote:
: > >
: > > my @answer = map { async { &_() } } @jobs;
: >
: > That still seems too explicit. I thought we had hyperoperators to
: > implictly parallelize for us:
: >
: > my @answer = @jobs.»();
: >
: > Which would run them in parallel automatically, if possible.
:
: Snazzy bit of syntactic shenanigans, that...and slick, if we're in that
: mode, but just to clarify, I *will* still be able to know whether or
: not something's going to thread? Or will it matter?

Shouldn't, if the hyperop is really promising that the calls won't
interact, and that's how hyper is currently defined. I think Best
Practices will not have hypers spawning huge interacting threads.

: Seems to me there will be times when I WANT a purely single-threaded

: system, even if I'm using hyperops -- in fact, especially if I'm using
: hyperops. Maybe we need a pragma to be able to step in and out as
: needed?
:
: { no threads;
: print @_.»();

: }

It seems a bit odd to use a construct for its syntactic sugar value but
take away its semantics...

If you just need ordering, this (or something like it) should serialize it:

print $_.() for @_;

Larry

[Paul Hodges]

LOL -- and "d'oh".
Apologies, not enough sleep.
I stand corrupted. :o]