asynchronous trigger_to_actor?

[arschles]

this thread forked from a larger thread: http://groups.google.com/group/pysage/t/3d28ed21fd90cd4f

currently, there are 2 general ways to send messages:

* trigger_to_actor: immediately sends a message via IPC to an actor.
blocks until the message send is complete
* queue_message_to_actor: queues a message to send to an actor for
later. returns immediately after the message was queued. once a
message is queued, it is the caller's responsibility to dequeue and
send it at a later time. this is done with the ActorMgr.tick method.

I'm proposing a third way, which provides functionality that
intersects the above 2 method. Called send_message for purposes of
this thread, this new method would queue the message immediately but
would not require the caller to dequeue and send the message at a
later time.

Here is a proposed spec for send_message:

* queues the message to send to the actor (regardless if that message
will be sent via IPC or over the wire)
* schedules that message to be sent ASAP (a background worker would be
responsible for doing the send)
* returns
* ensures that any given actor receives messages in order

Obviously, this message send functionality would be a poor choice to
use for high performance "real time" apps because it is non-
deterministic when the message is sent. Many non-realtime apps,
however, would trade high performance/control for convenience.

[John Yang]

On Sat, Apr 11, 2009 at 2:19 PM, arschles <arsc...@gmail.com> wrote:
>
> this thread forked from a larger thread: http://groups.google.com/group/pysage/t/3d28ed21fd90cd4f
>
> currently, there are 2 general ways to send messages:
>
> * trigger_to_actor: immediately sends a message via IPC to an actor.
> blocks until the message send is complete
> * queue_message_to_actor: queues a message to send to an actor for
> later. returns immediately after the message was queued. once a
> message is queued, it is the caller's responsibility to dequeue and
> send it at a later time. this is done with the ActorMgr.tick method.

Aaron, I think the documentation about how messages can be sent in
pysage is lacking. So I have created a page that hopefully outlines
this. Please refer to this and let me know if this clears things up.

http://code.google.com/p/pysage/wiki/SendingMessage

Basically, trigger is used only locally. And any "non-local" messages
are sent immediately.

>
> I'm proposing a third way, which provides functionality that
> intersects the above 2 method. Called send_message for purposes of
> this thread, this new method would queue the message immediately but
> would not require the caller to dequeue and send the message at a
> later time.
>
> Here is a proposed spec for send_message:
>
> * queues the message to send to the actor (regardless if that message
> will be sent via IPC or over the wire)
> * schedules that message to be sent ASAP (a background worker would be
> responsible for doing the send)

why not just pack it and send it? Where would the background worker
reside in, another thread?

> * returns
> * ensures that any given actor receives messages in order
>
> Obviously, this message send functionality would be a poor choice to
> use for high performance "real time" apps because it is non-
> deterministic when the message is sent. Many non-realtime apps,
> however, would trade high performance/control for convenience.

Actually, besides performance reasons, I can't think of a reason to do this.

> >
>

[arschles]

Thanks for the message documentation, John.

I'm still a bit worried about the synchronous nature of some of the
message sending calls. For instance, what if send_message is called by
a node when the network is down? Does it return a failure? And if so,
is the caller then responsible for re-sending? I think that having
asynchronous message sending functionality would eliminate a lot of
these questions (granted, it would raise others).

> why not just pack it and send it? Where would the background worker
> reside in, another thread?

I'd say that the async. versions could immediately pack it (and return
failure to the caller if it couldn't be packed), but shouldn't send it
because it's potentially unknown how long that operation will take.
The background worker would most likely reside in another thread.
Since the background worker is completely unseen by any actor, it can
be an implementation decision where the worker lives and how it works.

> Actually, besides performance reasons, I can't think of a reason to do this.

Perhaps we can revisit later? I think adding async. send functionality
(from a caller's perspective, queue_message without the requirement to
call tick later) could both improve perf (as you mentioned) of the
message sender and could make the system simpler from a user's
perspective (because there would be no need to call tick later, worry
about what happens when the network is down, etc...)

On Apr 11, 11:17 pm, John Yang <bigjh...@gmail.com> wrote:

[John Yang]

On Sun, Apr 12, 2009 at 1:34 PM, arschles <arsc...@gmail.com> wrote:
>
> Thanks for the message documentation, John.
>
> I'm still a bit worried about the synchronous nature of some of the
> message sending calls. For instance, what if send_message is called by
> a node when the network is down? Does it return a failure? And if so,
> is the caller then responsible for re-sending? I think that having
> asynchronous message sending functionality would eliminate a lot of
> these questions (granted, it would raise others).

I see where your concern is. The socket could block if the network is
down if the user chooses SOCKSTREAM. I think it is different with
datagram (UDP). Perhaps this is something we could work into the
network interface (being either a blocking/nonblocking/timeout
socket). The cost for actually sending the data in a normal
connection should be reasonable. If a user needs a process to remain
responsive, he could dedicate another process (pysage group) for just
sending network messages and have the GUI (for example) to reside in
its own group.

>
>> why not just pack it and send it?  Where would the background worker
>> reside in, another thread?
>
> I'd say that the async. versions could immediately pack it (and return
> failure to the caller if it couldn't be packed), but shouldn't send it
> because it's potentially unknown how long that operation will take.
> The background worker would most likely reside in another thread.
> Since the background worker is completely unseen by any actor, it can
> be an implementation decision where the worker lives and how it works.
>
>> Actually, besides performance reasons, I can't think of a reason to do this.
>
> Perhaps we can revisit later? I think adding async. send functionality
> (from a caller's perspective, queue_message without the requirement to
> call tick later) could both improve perf (as you mentioned) of the
> message sender and could make the system simpler from a user's
> perspective (because there would be no need to call tick later, worry
> about what happens when the network is down, etc...)

Just to be clear, the caller process would not need to call "tick", it
is the called process that would need to "tick" to receive that
message.

I'm not convinced of this either way yet. Let's give it some more
thoughts. But feel free to discuss this further.

Right now, I'd like to solidify the API and provide more documentation
for pysage.

John

[arschles]

> I see where your concern is. The socket could block if the network is
> down if the user chooses SOCKSTREAM. I think it is different with
> datagram (UDP). Perhaps this is something we could work into the
> network interface (being either a blocking/nonblocking/timeout
> socket). The cost for actually sending the data in a normal
> connection should be reasonable. If a user needs a process to remain
> responsive, he could dedicate another process (pysage group) for just
> sending network messages and have the GUI (for example) to reside in
> its own group.

could we have something like this built into pysage?

> Just to be clear, the caller process would not need to call "tick", it
> is the called process that would need to "tick" to receive that
> message.

I think I'm confused about the use of "tick". Based solely on the
sample code on the google code homepage, it looks like it's used in a
run-loop. Is that correct?

> I'm not convinced of this either way yet. Let's give it some more
> thoughts. But feel free to discuss this further.
>
> Right now, I'd like to solidify the API and provide more documentation
> for pysage.

Sounds good to me.

Cheers,
Aaron



On Apr 12, 9:34 pm, John Yang <bigjh...@gmail.com> wrote:

[John Yang]

On Mon, Apr 13, 2009 at 8:24 PM, arschles <arsc...@gmail.com> wrote:
>
>> I see where your concern is.  The socket could block if the network is
>> down if the user chooses SOCKSTREAM.  I think it is different with
>> datagram (UDP).  Perhaps this is something we could work into the
>> network interface (being either a blocking/nonblocking/timeout
>> socket).  The cost for actually sending the data in a normal
>> connection should be reasonable.  If a user needs a process to remain
>> responsive, he could dedicate another process (pysage group) for just
>> sending network messages and have the GUI (for example) to reside in
>> its own group.
>
> could we have something like this built into pysage?

Pysage already makes this pretty easy. Look at:

http://code.google.com/p/pysage/wiki/Grouping

>
>> Just to be clear, the caller process would not need to call "tick", it
>> is the called process that would need to "tick" to receive that
>> message.
>
> I think I'm confused about the use of "tick". Based solely on the
> sample code on the google code homepage, it looks like it's used in a
> run-loop. Is that correct?

Aaron, each process (pysage group) has its own actor manager
(singleton per process). When "tick" is called on the actor manager
on a particular process, only the messages on that particular process
are propagated.

So when process A sends a message to process B, process A does not
need to call "tick" because the message is sent to Process B's queue.
Therefore, process B needs to call "tick" on its own manager instance,
in order to distribute that received message to the actors resident in
process B.


Process A (send) --> Process B (tick) --> Actors in Process B (handle)

Does that help? Let me know because I would like to provide more
documentation around this area.

[arschles]

> Pysage already makes this pretty easy. Look at:
>
> http://code.google.com/p/pysage/wiki/Grouping

I looked at that documentation earlier. On the line:

<code>
# queues a message to the main group
mgr.queue_message_to_group(mgr.PYSAGE_MAIN_GROUP, SomeWorkDoneMessage
())
</code>

do the actors in the main group need to call tick in order to handle
SomeWorkDoneMessage?

> Does that help? Let me know because I would like to provide more
> documentation around this area.

yes, that helps a lot. maybe I could write some docs, as it would
definitely help my understanding. here's how I understand the message
sending infrastructure so far:

* an actor is the fundamental unit of concurrency
* one or more actors reside inside of processes, which are called
groups. by default, the main group is the python script that starts
the whole system (ie: the "main" function)
* each group has a single ActorMgr class, which any group-local actor
can use to receive messages, create new actors, send messages, receive
messages, etc...
* any single actor can communicate with a group on the same physical
machine (via an IPC mechanism) or with a group across a network (via
RakNet UDP, more transport protocols coming).
* It is the client's responsibility to know whether a message needs to
be sent over the network or over IPC
* It is every group's responsibility to explicitly receive messages.
each group does so with a call to the ActorMgr.tick method.

Is this correct? If so, does it comprehensively describe the message
passing infrastructure in pysage? I'm definitely willing to write docs
on this subject.

Aaron

On Apr 13, 9:34 pm, John Yang <bigjh...@gmail.com> wrote:

[John Yang]

On Mon, Apr 13, 2009 at 11:49 PM, arschles <arsc...@gmail.com> wrote:
>
>> Pysage already makes this pretty easy.  Look at:
>>
>> http://code.google.com/p/pysage/wiki/Grouping
>
> I looked at that documentation earlier. On the line:
>
> <code>
> # queues a message to the main group
> mgr.queue_message_to_group(mgr.PYSAGE_MAIN_GROUP, SomeWorkDoneMessage
> ())
> </code>
>
> do the actors in the main group need to call tick in order to handle
> SomeWorkDoneMessage?

Yes. The actormanager residing in the main group will need to call
"tick" to receive and process that message. This message is sent from
the "worker_group" process.

>
>> Does that help?  Let me know because I would like to provide more
>> documentation around this area.
>
> yes, that helps a lot. maybe I could write some docs, as it would
> definitely help my understanding. here's how I understand the message
> sending infrastructure so far:
>
> * an actor is the fundamental unit of concurrency

True in pure actor's model. In pysage, (without talking about
network), each process group is independent and therefore concurrent
with other groups. Many actors may reside in the same group, actors
in the same group are essentially synchronous.

> * one or more actors reside inside of processes, which are called
> groups. by default, the main group is the python script that starts
> the whole system (ie: the "main" function)
> * each group has a single ActorMgr class, which any group-local actor
> can use to receive messages, create new actors, send messages, receive
> messages, etc...
> * any single actor can communicate with a group on the same physical
> machine (via an IPC mechanism) or with a group across a network (via
> RakNet UDP, more transport protocols coming).

Right now, only the main group can communicate with all worker groups.
A worker group can communicate with the main group only. It's a tree
one level deep with the main group as the root, instead of a maximal
graph. Although, this could be enhanced in the future such that
worker groups can communicate with each other directly.

> * It is the client's responsibility to know whether a message needs to
> be sent over the network or over IPC

Yes. You need to be aware whether you are working with a group or
with a network. Although, this can be abstracted away later.

> * It is every group's responsibility to explicitly receive messages.
> each group does so with a call to the ActorMgr.tick method.

Yes. Each "tick" call polls for IPC as well as network packets.
Although, like we talked about, the actors update should be broken out
from tick, or passed in as a parameter.

>
> Is this correct? If so, does it comprehensively describe the message
> passing infrastructure in pysage? I'm definitely willing to write docs
> on this subject.

Yeap, mostly correct. I added some clarifications to some points.
Aaron, it would be a great help if you could contribute to the
documentation. I'm currently using google code wiki as the
documentation tool. I'm not sure if it's the best approach, but we
may have to work with it. Any thoughts on that?

[John Yang]

Right. This is an important point. Actor Manager instance is a
singleton per process. It is responsible for receiving, propagating
messages for actors in that process. It can only be used to
create/manage actors in that process.

[arschles]

Hey John,

When I'm finished with the UDP transport layer, I'll write some docs
on this subject. I think the google code wiki is a good place to put
docs as well. I don't seem to have access to creating and editing wiki
pages, though. Can you give me access?

Aaron

On Apr 15, 9:08 am, John Yang <bigjh...@gmail.com> wrote:

> ...
>
> read more »

[John Yang]

Aaron:

I added you to project members. You should have access to the wiki now.

John