Event design sketch
Dan Sugalski
about this.
I'll get to IO after, since the two are pretty tightly intertwined, and
changes to this will affect the IO stuff too.
------------>Snip here<---------------
Events, another design sketch
Overview
========
Events are a terrific idea, as evidenced by the vast array of event
handling libraries there are. (There are at least a half dozen on
CPAN) All of which are the same but different, and incompatible. Of
course.
Proper event handling's the only way to manage asynchronous IO, and
since parrot needs event handling it only makes sense to expose it in
general, so we can avoid the problems of having multiple event
handling loops.
Event basics
============
An event is an indication to your program that something has
happened. It may be that a timer has fired, a packet has been
received (or sent) on a socket, a disk read has completed, or that
the user has moved the mouse.
Events are asynchronous--that is they can happen at any time, and
your program has no real control over when they are generated. It
does, however, have control over when they are processed. More on
that later.
Event Classes
=============
There are two main classes of events, which we'll call expected and
unexpected.
An expected event is one that your program is specifically expecting
to get as a result of a request for asynchronous action, and each
expected event has a corresponding event handle. User code can wait
for an expected event to complete, cancel an expected event, and
query the status of an expected event. Expected events may also be
either complete or incomplete. An incomplete event is one whose
requested action hasn't yet happened, while a complete event
Unexpected events, on the other hand, happen for reasons outside of
your program. Signals, GUI events, and exceptions for asynchronous
files (which may not be delivered as signals on many platforms) are
examples of unexpected events. Unexpected events, by their nature, are
always complete, since they aren't generated until an action actually
occurs.
Some event sources may be either expected or unexpected. Network
connections are an example of this--you may have a network filehandle
set either as an expected event source, where your program must
explicitly ask for data, or set as an unexpected event source, where
a monitoring thread captures data as it is received and generates
events for the captured data.
Callbacks and User Data
=======================
Each expected event can have a callback routine and piece of user
data associated with it. When the event request is satisfied (The IO
operation completes or the timer fires, for example) the callback
associated with it is invoked. The user data, as well as the event
data, if there is any, is passed into the callback.
Callback signatures are fixed, and of the form:
(Preturndata, Icommandstatus) =
callbacksub(Pevent, Peventdata, Puserdata, Icommand)
The callback is passed in the event PMC, the PMC for the event data,
and the PMC for the user data, if any. (Either or both of these can
be the Null PMC if there is no user or event data) Command is always
0 for callback handling. (Callbacks and IO layer functions are
identical. More detail in the IO section)
The callback must then return the data and a command status. The
status should be 0 for correct completion, a non-zero negative number
for an error, and a non-zero positive number for successful
completion with a change in status for the next handler in the chain,
if there is one.
If a callback function throws an exception the exception itself is
deferred. Expected events defer the exception until the event is
waited on, while unexpected events throw their excecption when their
event functions are done processing. (This does mean that expected
events may have their exceptions effectively ignored if the event is
never waited on)
When there are multiple event handler in a chain, each handler can
alter the data and command passed to the next handler in the
chain. The return data is passed in as the user data to the next
command, and the return status is, if a positive non-zero number,
passed to the next handler in the chain. If the return status is 0,
then the same command is passed into the next handler in the
chain. This allows event handlers to preprocess the event if need be.
Generic Handlers
================
Each event source can have a generic handler associated with it. This
handler is called before the event's specific callback is invoked, if
it has one. Generic handlers have the same function signature as
callback functions do, and behave the same way--if they return 0 the
event is passed to the next handler in the chain, and if they return
non-zero then the event stops passing through and is marked as failed.
Note that the handler table is scoped, however due to the inherently
unpredictable nature of asynchronous events it isn't possible to
guarantee that changes in the handler table due to scope changes will
take place immediately so far as event sources are concerned--that
is, the program may leave a scope and change the handler table yet
have an event generated that uses the old handler table.
Events and Exceptions
=====================
Event handlers are generally discouraged from throwing exceptions,
however it is sometimes necessary. Exceptions are handled differently
by expected and unexpected events.
With an expected event, if an event handler is terminated by an
unhandled exception, that exception is attached to the event itself,
and when user-level code C<wait>s on the event, the exception is
thrown then.
Unexpected events whose handlers throw exceptions are handled somewhat
differently. In that case, the exception is thrown as soon as the
event is drained from the event queue. This may or may not happen
immediately, depending on how the event in question is currently being
handled. If the event is processed in a separate event handling thread
then the event is put in the main thread's event queue and the
exception will be thrown when the main thread drains the queue and
hits the event. If the event is currently being processed in the main
thread (because there is no event-handling thread, or this event
isn't being handled by one) then as soon as the event is drained from
the event queue the exception is thrown.
How events are handled
======================
There are three steps in event handling.
First, an event is triggered. This puts the event into the event
queue, but otherwise does nothing. (This operation is interrupt-safe,
though there issues in obtaining event PMCs at interrupt level in
some cases)
Next, a queue runner pulls an event out of the event queue and calls
its handlers. These are a combination of any event class handlers as
well as an event-specific callback, if there is one. This operation
happens as normal user-level code, though it may be done
significantly after the event itself is put in the queue, and may not
happen in the same thread as the event was requested from. [NB: I'm
not sure what to do about event handling threads and shared data yet]
Finally, for expected events, something can wait on the event to get
the event's data and status. This can be done before the event has
completed, in which case the operation will wait (draining the event
queue of pending events)
Note that the queue runner will automatically wait on any unexpected
event that it completes. This puts the event in the done state and
will throw any exception that the event handlers may have
generated. Expected events, however, will be left alone until
user-level code explicitly waits on it.
Event States
============
Events have five states. They are:
* Quiescent - The event object has been newly allocated but hasn't
been associated with anything yet and can't be triggered
* Waiting - The operation the event triggers on hasn't yet happened
* Triggered - The operation the event triggers on has happened, but
the event's handlers haven't been called yet
* Ready - The event has been processed by its handlers but hasn't yet
been waited on to flush out any exception it might have.
* Done - The event has flushed any pending exceptions and can now
only be queried for its status and data
Creating Events
===============
Events are just PMCs, and there are no ops specifically to create
them. To create an event, allocate a PMC of the class Event. These
are array/hash combination PMCs and may be accessed as follows:
0 (CLASS) - The class of the event
1 (EVENT_DATA) - The event data associated with this event
2 (USER_DATA) - The user data associated with the event
3 (CALLBACK) - The callback sub for this event
4 (EXCEPTION) - The pending exception for this event, if any
5 (STATUS) - The status from the exception handlers
6 (STATE) - What state the event is in
7 (LEVEL) - The event level this event is at
Event Opcodes
=============
The opcodes in this section are a combination of event requests and
event handling ops. It doesn't include the IO ops--those are separate.
Most of the event request ops have two forms, one of which takes a
callback PMC and user data PMC.
checkevent
Explicitly check to see if there are any events pending in the event queue and,
if so process one.
drainqueue
Wait forever, draining the event queue as it fills. This op only
exits if it encounters a QUEUE_DONE event in the queue.
wait Pevent
Wait for the specified event to complete. If the event has already
completed the op doesn't wait. If there is a pending exception it is
thrown. This opcode will drain the pending event queue.
getstat Istatus, Pevent
Returns the status of the event, noting whether it has been completed
yet and, if so, what happened.
getdata Pdata, Pevent
getdata Sdata, Pevent
Get the data associated with the event, if there is any. What the data is
depends on what generated the event. (For filehandle reads this will
be the data read from the filehandle)
getuser Puserdata, Pevent
Get the user data that was passed into this event.
cancel Pevent
Cancel the event, if possible.
post Pevent
Post an event to the event queue for later handling
sethandler Ieventclass, Phandlersub
Set the default handler for the specified class of events.
seteventlevel Ieventlevel
Set the current event level. Events of a lower level than this won't
be processed.
settimer Pevent, Nseconds[, Pcallback, Puserdata]
settimer Pevent, Iseconds[, Pcallback, Puserdata]
Set a timer to go off in C<seconds> seconds.
setalarm Pevent, Nabs_time[, Pcallback, Puserdata]
setalarm Pevent, Iabs_time[, Pcallback, Puserdata]
Set a timer to go off at the absolute time specified.
setinterval Ninterval_seconds[, Pcallback, Puserdata]
setinterval Iinterval_seconds[, Pcallback, Puserdata]
Set an interval timer that goes off every interval_second seconds.
When are events handled?
========================
Parrot does C<not> guarantee immediate delivery of events. That,
unfortunately, is untenable for us, as much as we would like it. The
overhead involved in checking for pending events every op is
excessive for at least some of the run cores.
Instead, parrot guarantees that events get checked with some
regularity. All ops that appear to block are actually waiting for a
wakeup event, and so drain the queue as they wait. Other ops may check
the event queue, and are tagged with the "check_event" property in the
ops files. It's the compiler's job to ensure that at least some
event-checking ops get executed, unfortunately.
C Interface
===========
Because many events are actually generated from within C code, the
following API is exposed for use:
INTVAL Parrot_ping_event(Parrot_Interp Interpreter, INTVAL type);
Note that an event of type C<type> has just occurred. Returns either
0 on success or 1 on failure. This function may fail if the target
interpreter is unable to post an event to its event queue. This
normally happens because there are no event PMCs available to
allocate for the event.
INTVAL Parrot_post_event(Parrot_Interp Interpreter, PMC *event);
Post the event PMC to the target interpreter's event queue. Returns
success status, 0 on success, 1 on failure. This function may fail if
the event can't be posted to the target interpreter's event queue for
some reason.
--
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Simon Glover
One quick point: unless I'm misunderstanding something, there seems to
be no easy way to wait on multiple events to complete if those events
can occur in any order. For instance, suppose we have 5 events, P1 - P5
If we know that the events will occur in some specified order (say
1-2-3-4-5), then we can just loop over C<wait Px>, specifying the
apporpriate x for each iteration. However, if the events actually occur
in the order 4-3-2-5-1, we'll drop events 2-5 while waiting for 1.
Now, this is fine if 1 _must_ happen before 2 etc., but often this is
more restrictive than we need to be, since all we really care about is
ensuring that all 5 events are handled before we do anything else.
Is there any easy and efficient way to do this with the current
framework?
Simon
Dan Sugalski
There's no easy way, no. The best you could do is have the code:
wait E1
wait E2
wait E3
wait E4
wait E5
which'll work (and everything'll get processed properly, and as soon
as it's available) but is somewhat nasty.
Thinking we might want:
waitall Parray_of_events
waitany Parray_of_events
?
Uri Guttman
DS> Event Classes
DS> =============
DS> There are two main classes of events, which we'll call expected and
DS> unexpected.
DS> An expected event is one that your program is specifically expecting
DS> to get as a result of a request for asynchronous action, and each
DS> expected event has a corresponding event handle. User code can wait
DS> for an expected event to complete, cancel an expected event, and
DS> query the status of an expected event. Expected events may also be
DS> either complete or incomplete. An incomplete event is one whose
DS> requested action hasn't yet happened, while a complete event
^^^^^
?????
DS> Unexpected events, on the other hand, happen for reasons outside of
DS> your program. Signals, GUI events, and exceptions for asynchronous
DS> files (which may not be delivered as signals on many platforms) are
DS> examples of unexpected events. Unexpected events, by their nature, are
DS> always complete, since they aren't generated until an action actually
DS> occurs.
DS> Some event sources may be either expected or unexpected. Network
DS> connections are an example of this--you may have a network filehandle
DS> set either as an expected event source, where your program must
DS> explicitly ask for data, or set as an unexpected event source, where
DS> a monitoring thread captures data as it is received and generates
DS> events for the captured data.
i disagree with the two classes. how can you handle an unexpected event
without know it could happen? effectively all events are expected, some
are more expected than others. :)
the only way to get mouse movements (which is really translated to some
i/o or socket event by the windowing system) is to ask for them. same
with signals. if you don't ask to handle a signal, it uses the default
(usually IGNORE or whatever). you need a handler for each possible event
so effectively all events you can handle are expected.
DS> Callbacks and User Data
DS> =======================
DS> Each expected event can have a callback routine and piece of user
DS> data associated with it. When the event request is satisfied (The IO
DS> operation completes or the timer fires, for example) the callback
DS> associated with it is invoked. The user data, as well as the event
DS> data, if there is any, is passed into the callback.
the event handle(r) itself also is passed to the callback. you say event
data there and maybe you mean event structure/object?
so let me clarify. the callback is passed the event itself, optionally
any data read for the event, and optionally any private data passed into
the event (this is usually the object or user id for this event).
DS> Callback signatures are fixed, and of the form:
DS> (Preturndata, Icommandstatus) =
DS> callbacksub(Pevent, Peventdata, Puserdata, Icommand)
DS> The callback is passed in the event PMC, the PMC for the event data,
DS> and the PMC for the user data, if any. (Either or both of these can
DS> be the Null PMC if there is no user or event data) Command is always
DS> 0 for callback handling. (Callbacks and IO layer functions are
DS> identical. More detail in the IO section)
how about renaming userdata to iodata since that is what it
is. eventdata could be left alone or actually renamed to userdata since
it is the data passed in to the event call and the user expects it to
come back so the code can identify the event and associate it with an
object/structure.
DS> If a callback function throws an exception the exception itself is
DS> deferred. Expected events defer the exception until the event is
DS> waited on, while unexpected events throw their excecption when their
DS> event functions are done processing. (This does mean that expected
DS> events may have their exceptions effectively ignored if the event is
DS> never waited on)
you need to clarify expected vs unexpected more. i wrote above they
should all be expected. i don't see how this section would change that.
DS> Events and Exceptions
DS> =====================
DS> Event handlers are generally discouraged from throwing exceptions,
DS> however it is sometimes necessary. Exceptions are handled differently
DS> by expected and unexpected events.
DS> With an expected event, if an event handler is terminated by an
DS> unhandled exception, that exception is attached to the event itself,
DS> and when user-level code C<wait>s on the event, the exception is
DS> thrown then.
huh? if you are in an event handler, you have seen the event (and are
presumably in user level code), so why would you wait for it again?
DS> Unexpected events whose handlers throw exceptions are handled somewhat
DS> differently. In that case, the exception is thrown as soon as the
DS> event is drained from the event queue. This may or may not happen
DS> immediately, depending on how the event in question is currently being
DS> handled. If the event is processed in a separate event handling thread
DS> then the event is put in the main thread's event queue and the
DS> exception will be thrown when the main thread drains the queue and
DS> hits the event. If the event is currently being processed in the main
DS> thread (because there is no event-handling thread, or this event
DS> isn't being handled by one) then as soon as the event is drained from
DS> the event queue the exception is thrown.
DS> How events are handled
DS> ======================
DS> There are three steps in event handling.
DS> First, an event is triggered. This puts the event into the event
DS> queue, but otherwise does nothing. (This operation is interrupt-safe,
DS> though there issues in obtaining event PMCs at interrupt level in
DS> some cases)
DS> Next, a queue runner pulls an event out of the event queue and calls
DS> its handlers. These are a combination of any event class handlers as
DS> well as an event-specific callback, if there is one. This operation
DS> happens as normal user-level code, though it may be done
DS> significantly after the event itself is put in the queue, and may not
DS> happen in the same thread as the event was requested from. [NB: I'm
DS> not sure what to do about event handling threads and shared data yet]
DS> Finally, for expected events, something can wait on the event to get
DS> the event's data and status. This can be done before the event has
DS> completed, in which case the operation will wait (draining the event
DS> queue of pending events)
DS> Note that the queue runner will automatically wait on any unexpected
DS> event that it completes. This puts the event in the done state and
DS> will throw any exception that the event handlers may have
DS> generated. Expected events, however, will be left alone until
DS> user-level code explicitly waits on it.
so you separate expected vs unexpected as really user level handlers vs
parrot internal handlers. why not call them all expected and
internal. only those with user level stuff get a user level callback or
wakeup on a wait. i don't see why the handling is so different.
DS> Event States
DS> ============
DS> Events have five states. They are:
DS> * Quiescent - The event object has been newly allocated but hasn't
DS> been associated with anything yet and can't be triggered
call that 'new' instead. or maybe disabled (but that means the event has
been associated with something already so new is better)?
DS> * Waiting - The operation the event triggers on hasn't yet
DS> happened
this will be confusing with the 'wait' operation. i would call it
'watching' (from event.pm) or something similar (maybe 'active' or
'enabled'?)
DS> * Triggered - The operation the event triggers on has happened, but
DS> the event's handlers haven't been called yet
DS> * Ready - The event has been processed by its handlers but hasn't yet
DS> been waited on to flush out any exception it might have.
DS> * Done - The event has flushed any pending exceptions and can now
DS> only be queried for its status and data
you also can use a 'stopped/disabled' state. in many cases you want an active
event but stopped for a while. a good example is a write event on a
an empty socket. if enabled, it will trigger all the time since the
socket is always writeable and that will suck up the cpu. what i do is
make write events disabled by default and you enable then when you have
data to actually write. when the user space write buffer is empty you
disable the event again. this is much cleaner than creating/destroying
events.
so my state list is:
new
enabled
disabled
triggered
ready
done
when an event is done, who/what flushes the event itself from the
system, normal GC?
DS> getdata Pdata, Pevent
DS> getdata Sdata, Pevent
DS> Get the data associated with the event, if there is any. What the
DS> data is depends on what generated the event. (For filehandle reads
DS> this will be the data read from the filehandle)
but won't that data already be passed in the callback? another point is
that event i/o data should be passed by reference and treated as
readonly to save copying all over the place.
DS> cancel Pevent
DS> Cancel the event, if possible.
add enable/disable here.
DS> settimer Pevent, Nseconds[, Pcallback, Puserdata]
DS> settimer Pevent, Iseconds[, Pcallback, Puserdata]
DS> Set a timer to go off in C<seconds> seconds.
DS> setalarm Pevent, Nabs_time[, Pcallback, Puserdata]
DS> setalarm Pevent, Iabs_time[, Pcallback, Puserdata]
DS> Set a timer to go off at the absolute time specified.
DS> setinterval Ninterval_seconds[, Pcallback, Puserdata]
DS> setinterval Iinterval_seconds[, Pcallback, Puserdata]
DS> Set an interval timer that goes off every interval_second seconds.
are all those times (in Nseconds) are floats with fractional seconds?
i don't think there is a need for all those variants. why would alarm
need any special opcode when it is just a timer with a delay of abs_time
- NOW? let the coder handle that and lose the extra op codes.
also the interval can be folded in as a extra arg to the timer (either a
repeat flag or an interval). the initial seconds can be the first delay
and then reused for intervals or the interval value can take over:
settimer Pevent, Idelay_seconds, Iinterval_seconds[, Pcallback,
Puserdata]
if interval_seconds is set (>0), use it for a repeating timer. if
delay_seconds is set (>0), it is the first interval.
so now all you have is two signatures (for float or integer seconds).
you could have a mix of float delay and integer interval but that just
makes 4 signatures.
DS> C Interface
DS> ===========
DS> Because many events are actually generated from within C code, the
DS> following API is exposed for use:
DS> INTVAL Parrot_ping_event(Parrot_Interp Interpreter, INTVAL type);
DS> Note that an event of type C<type> has just occurred. Returns either
DS> 0 on success or 1 on failure. This function may fail if the target
DS> interpreter is unable to post an event to its event queue. This
DS> normally happens because there are no event PMCs available to
DS> allocate for the event.
clarify. who 'notes'? maybe call it 'check_event'? how is this different
than post_event?
overall a good start. looking forward to seeing a clean fast event core.
uri
--
Uri Guttman ------ u...@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
Chromatic
> Thinking we might want:
>
> waitall Parray_of_events
> waitany Parray_of_events
>
> ?
I certainly do!
I'm also curious how to write an interface to an existing event system.
Being able to write it all in PASM is a bonus.
-- c
Uri Guttman
SG> One quick point: unless I'm misunderstanding something, there seems to
SG> be no easy way to wait on multiple events to complete if those events
SG> can occur in any order. For instance, suppose we have 5 events, P1 - P5
SG> If we know that the events will occur in some specified order (say
SG> 1-2-3-4-5), then we can just loop over C<wait Px>, specifying the
SG> apporpriate x for each iteration. However, if the events actually occur
SG> in the order 4-3-2-5-1, we'll drop events 2-5 while waiting for 1.
SG> Now, this is fine if 1 _must_ happen before 2 etc., but often this is
SG> more restrictive than we need to be, since all we really care about is
SG> ensuring that all 5 events are handled before we do anything else.
SG> Is there any easy and efficient way to do this with the current
SG> framework?
you have a callback for each event and it sets a flag (better idea
below). when all the wanted flags are set, you post another event that
all 5 things are done. your main code waits on that event.
a nice way to handle the multiple flag things is with a hash and a
callback. you preset the hash to all the keys you are waiting on. when each
handler is triggered, it deleted its key (which can come from the
private event data stored in the event itself) from the hash. when the
hash is empty, trigger the callback. i call this a scatter (start the
async actions) and gather (collect their done flags) operation.
this is best done at a user level. let parrot handle each event
individually. there is no major benefit to parrot doing this for you IMO
as it is not system level at all.
Dan Sugalski
>On Tue, 2004-05-11 at 09:44, Dan Sugalski wrote:
>
>> Thinking we might want:
>>
>> waitall Parray_of_events
>> waitany Parray_of_events
>>
>> ?
>
>I certainly do!
Fair enough.
>I'm also curious how to write an interface to an existing event system.
>Being able to write it all in PASM is a bonus.
I don't think it can be all-PASM, except maybe (and maybe not...)
with a separate thread for the existing event source. To do it in all
pasm means calling back into parrot from interrupt level, which isn't
really doable, or have a thread just waiting on events from the
alternate event system to post into the parrot event queue.
Which, I suppose, is doable, though there are still those pesky
interrupt-level limitations pontentially in there. We'll need to get
shared-pool threads working better before we can really take a shot
at it.
Chromatic
> >I'm also curious how to write an interface to an existing event system.
> >Being able to write it all in PASM is a bonus.
>
> I don't think it can be all-PASM, except maybe (and maybe not...)
> with a separate thread for the existing event source. To do it in all
> pasm means calling back into parrot from interrupt level, which isn't
> really doable, or have a thread just waiting on events from the
> alternate event system to post into the parrot event queue.
Another approach may be to expose the PollEvent and WaitEvent functions
to the event system as alternate sources of events. If I can do that
from PASM, I think I'm okay.
-- c
Dan Sugalski
That'll still need some C. The event system as it stands is all
active--all event sources put events into the system, rather than
having the event system go looking at event sources for events. You'd
either need to queue up regular timer events to go check for new
stuff or have a thread actively polling the source for events and
throwing them into parrot's event queue.
Uri Guttman
c> On Tue, 2004-05-11 at 10:24, Dan Sugalski wrote:
>> >I'm also curious how to write an interface to an existing event system.
>> >Being able to write it all in PASM is a bonus.
>>
>> I don't think it can be all-PASM, except maybe (and maybe not...)
>> with a separate thread for the existing event source. To do it in all
>> pasm means calling back into parrot from interrupt level, which isn't
>> really doable, or have a thread just waiting on events from the
>> alternate event system to post into the parrot event queue.
c> Another approach may be to expose the PollEvent and WaitEvent functions
c> to the event system as alternate sources of events. If I can do that
c> from PASM, I think I'm okay.
i don't think exposing them is a good idea as they can be broken by user
stuff. and see my other reply for another solution to waiting on
multiple events without any extra core code. it is very easy and
eliminates the need for a group wait op. one problem with a group wait
is that you can ONLY wait for that group. what if you wanted to know
when group A events are all done and also when group B events are all
done? and what if you have other single events floating around? rarely
do you just have a single group of events to wait for.
my solution works with any combination of events and groups.
Dan Sugalski
> >>>>> "DS" == Dan Sugalski <d...@sidhe.org> writes:
>
> DS> Event Classes
> DS> =============
>
> DS> There are two main classes of events, which we'll call expected and
> DS> unexpected.
>
> DS> An expected event is one that your program is specifically expecting
> DS> to get as a result of a request for asynchronous action, and each
> DS> expected event has a corresponding event handle. User code can wait
> DS> for an expected event to complete, cancel an expected event, and
> DS> query the status of an expected event. Expected events may also be
> DS> either complete or incomplete. An incomplete event is one whose
> DS> requested action hasn't yet happened, while a complete event
> ^^^^^
>?????
Just chop that last sentence fragment out. It makes no sense. :)
>
> DS> Unexpected events, on the other hand, happen for reasons outside of
> DS> your program. Signals, GUI events, and exceptions for asynchronous
> DS> files (which may not be delivered as signals on many platforms) are
> DS> examples of unexpected events. Unexpected events, by their nature, are
> DS> always complete, since they aren't generated until an action actually
> DS> occurs.
>
> DS> Some event sources may be either expected or unexpected. Network
> DS> connections are an example of this--you may have a network filehandle
> DS> set either as an expected event source, where your program must
> DS> explicitly ask for data, or set as an unexpected event source, where
> DS> a monitoring thread captures data as it is received and generates
> DS> events for the captured data.
>
>i disagree with the two classes. how can you handle an unexpected event
>without know it could happen? effectively all events are expected, some
>are more expected than others. :)
The terminology there's a bit strained, and I think it's in large
part responsible for most of the rest of the confusion.
They're probably better called Named and Anonymous events, though
that's a bit dodgy in some ways too. Maybe specific and generic
events would be better.
The basic difference is that with Named events you *know* in advance
that the event is coming because you explicitly requested it by
setting a timer or making an IO request. (A specific timer or IO
request, not timer or IO requests in general) This is different from
things like signals as you've not asked for a particular signal to
fire--they fire off whenever something outside your control and
expectation happens.
Because of this, you have the event PMC for a Named event before the
event occurs and thus can wait on it. You *don't* have the event PMC
for an anonymous event, so you can't wait on it, all you can do is
semi-generically react once it's occurred.
> DS> Callbacks and User Data
> DS> =======================
>
> DS> Each expected event can have a callback routine and piece of user
> DS> data associated with it. When the event request is satisfied (The IO
> DS> operation completes or the timer fires, for example) the callback
> DS> associated with it is invoked. The user data, as well as the event
> DS> data, if there is any, is passed into the callback.
>
>the event handle(r) itself also is passed to the callback.
No, it isn't.
>you say event
>data there and maybe you mean event structure/object?
Nope, I mean event data. It's the data that whatever triggered the
event generated.
For mouse click events it may be the X/Y/Duration of the click, for
read requests it's the data read, and so on.
>so let me clarify. the callback is passed the event itself, optionally
>any data read for the event, and optionally any private data passed into
>the event (this is usually the object or user id for this event).
Yes.
> DS> Callback signatures are fixed, and of the form:
>
> DS> (Preturndata, Icommandstatus) =
> DS> callbacksub(Pevent, Peventdata, Puserdata, Icommand)
>
> DS> The callback is passed in the event PMC, the PMC for the event data,
> DS> and the PMC for the user data, if any. (Either or both of these can
> DS> be the Null PMC if there is no user or event data) Command is always
> DS> 0 for callback handling. (Callbacks and IO layer functions are
> DS> identical. More detail in the IO section)
>
>how about renaming userdata to iodata since that is what it
>is.
It isn't though. The event data is the IO data here.
> DS> Events and Exceptions
> DS> =====================
>
> DS> Event handlers are generally discouraged from throwing exceptions,
> DS> however it is sometimes necessary. Exceptions are handled differently
> DS> by expected and unexpected events.
>
> DS> With an expected event, if an event handler is terminated by an
> DS> unhandled exception, that exception is attached to the event itself,
> DS> and when user-level code C<wait>s on the event, the exception is
> DS> thrown then.
>
>huh? if you are in an event handler, you have seen the event (and are
>presumably in user level code), so why would you wait for it again?
You're confused here. The event handler is what throws the exception.
It's the user code that's waited on the event that gets the
exception. (And it's possible, though likely a bad idea, to have an
exception handler make an async request and wait for it to complete)
> DS> Event States
> DS> ============
>
> DS> Events have five states. They are:
>
> DS> * Quiescent - The event object has been newly allocated but hasn't
> DS> been associated with anything yet and can't be triggered
>
>call that 'new' instead. or maybe disabled (but that means the event has
>been associated with something already so new is better)?
>
> DS> * Waiting - The operation the event triggers on hasn't yet
> DS> happened
>
>this will be confusing with the 'wait' operation. i would call it
>'watching' (from event.pm) or something similar (maybe 'active' or
>'enabled'?)
>
> DS> * Triggered - The operation the event triggers on has happened, but
> DS> the event's handlers haven't been called yet
> DS> * Ready - The event has been processed by its handlers but hasn't yet
> DS> been waited on to flush out any exception it might have.
> DS> * Done - The event has flushed any pending exceptions and can now
> DS> only be queried for its status and data
>
>you also can use a 'stopped/disabled' state. in many cases you want an active
>event but stopped for a while.
That can't happen. You can cancel an event, but that's it. You can
stop an event source, but that's different.
>when an event is done, who/what flushes the event itself from the
>system, normal GC?
Yep.
> DS> getdata Pdata, Pevent
> DS> getdata Sdata, Pevent
>
> DS> Get the data associated with the event, if there is any. What the
> DS> data is depends on what generated the event. (For filehandle reads
> DS> this will be the data read from the filehandle)
>
>but won't that data already be passed in the callback?
Yes. Which doesn't do you much good if you're not in the callback... :)
> DS> cancel Pevent
>
> DS> Cancel the event, if possible.
>
>add enable/disable here.
Nope. Events aren't enable-able/disable-able.
> DS> settimer Pevent, Nseconds[, Pcallback, Puserdata]
> DS> settimer Pevent, Iseconds[, Pcallback, Puserdata]
>
> DS> Set a timer to go off in C<seconds> seconds.
>
> DS> setalarm Pevent, Nabs_time[, Pcallback, Puserdata]
> DS> setalarm Pevent, Iabs_time[, Pcallback, Puserdata]
>
> DS> Set a timer to go off at the absolute time specified.
>
> DS> setinterval Ninterval_seconds[, Pcallback, Puserdata]
> DS> setinterval Iinterval_seconds[, Pcallback, Puserdata]
>
> DS> Set an interval timer that goes off every interval_second seconds.
>
>are all those times (in Nseconds) are floats with fractional seconds?
Yes.
>i don't think there is a need for all those variants. why would alarm
>need any special opcode when it is just a timer with a delay of abs_time
>- NOW? let the coder handle that and lose the extra op codes.
I didn't see any reason to not do absolute times as well as deltas.
It's no big deal either way.
>also the interval can be folded in as a extra arg to the timer (either a
>repeat flag or an interval). the initial seconds can be the first delay
>and then reused for intervals or the interval value can take over:
>
> settimer Pevent, Idelay_seconds, Iinterval_seconds[, Pcallback,
> Puserdata]
>
>if interval_seconds is set (>0), use it for a repeating timer. if
>delay_seconds is set (>0), it is the first interval.
Ick. No. It's not like there's any less code there, and the intent's
a bit clearer.
> DS> C Interface
> DS> ===========
>
> DS> Because many events are actually generated from within C code, the
> DS> following API is exposed for use:
>
> DS> INTVAL Parrot_ping_event(Parrot_Interp Interpreter, INTVAL type);
>
> DS> Note that an event of type C<type> has just occurred. Returns either
> DS> 0 on success or 1 on failure. This function may fail if the target
> DS> interpreter is unable to post an event to its event queue. This
> DS> normally happens because there are no event PMCs available to
> DS> allocate for the event.
>
>clarify. who 'notes'?
Posts an event of the specified type to the event queue of the target
interpreter.
> maybe call it 'check_event'? how is this different
>than post_event?
No PMC handling. It means that whatever code's doing it doesn't have
to manage a PMC queue (since if it's in an interrupt handler it can't
allocate one--they need to be preallocated) and set up all the bits
of the event PMC. Of limited utility, but for things like signal
handlers that can't do much anyway and have so little information
available to them it's sufficient.
Chromatic
> That'll still need some C. The event system as it stands is all
> active--all event sources put events into the system, rather than
> having the event system go looking at event sources for events. You'd
> either need to queue up regular timer events to go check for new
> stuff or have a thread actively polling the source for events and
> throwing them into parrot's event queue.
The thread seems like the way to go, if I can enqueue an event without
writing any C -- just post EventPMC from PASM?
So for SDL, I'd start a separate thread that blocks on SDL_WaitEvent,
creating and posting events when they happen. My main program would
handle the events as normal Parrot events. Standard producer consumer
stuff.
Since it's blocking, it won't eat up too many resources -- that's nice.
It'd be nice to have the SDL event thread ignore events I don't care
about though, instead of creating event PMCs I'll just throw away later.
Is this what you have in mind?
-- c
Dan Sugalski
You can always Get Horribly Clever in the event handling thread and
look at what the SDL library's handed you. If it's uninteresting you
can just throw it away rather than creating an event to be discarded.
>Is this what you have in mind?
Yep.
Chromatic
> >Since it's blocking, it won't eat up too many resources -- that's nice.
> >It'd be nice to have the SDL event thread ignore events I don't care
> >about though, instead of creating event PMCs I'll just throw away later.
>
> You can always Get Horribly Clever in the event handling thread and
> look at what the SDL library's handed you. If it's uninteresting you
> can just throw it away rather than creating an event to be discarded.
What's interesting in one context is quite dull in another. This means
shuttling data between threads, which is always the point at which
things start to go wrong*. At least for me.
-- c
* Since we haven't had a good flamewar in at least twenty minutes,
"Threads are good as long as you keep them far away from each other!"
Gordon Henriksen
> At 10:33 AM -0700 5/11/04, chromatic wrote:
> >On Tue, 2004-05-11 at 10:24, Dan Sugalski wrote:
> >
> >> >I'm also curious how to write an interface to an
> existing event system.
> >> >Being able to write it all in PASM is a bonus.
> >>
> >> I don't think it can be all-PASM, except maybe (and maybe not...)
> >> with a separate thread for the existing event source. To
> do it in all
> >> pasm means calling back into parrot from interrupt level,
> which isn't
> >> really doable, or have a thread just waiting on events from the
> >> alternate event system to post into the parrot event queue.
> >
> >Another approach may be to expose the PollEvent and
> WaitEvent functions
> >to the event system as alternate sources of events. If I can do that
> >from PASM, I think I'm okay.
>
> That'll still need some C. The event system as it stands is all
> active--all event sources put events into the system, rather than
> having the event system go looking at event sources for events. You'd
> either need to queue up regular timer events to go check for new
> stuff or have a thread actively polling the source for events and
> throwing them into parrot's event queue.
What's the plan for integrating with system events, then? Mac OS X and
Windows both have robust, irreplacable, system-managed event loops.
parrot's loop can (and should) run in a parallel thread to those, but
certainly can't presume to take over entirely. It simply can't work.
If you're calling this an event system, it ought to mesh in with the
notion of events that every programmer who uses them will have:
- Mouse events.
- Keyboard events.
- Redraw events.
- Drag-'n'-drop events.
- Menu command events.
- etc., etc., etc..
Problems in this domain also include: Focus management, propagation
along the responder chain, enabling/disabling commands, default event
handlers.
You won't get any of these events from an I/O wait on Win32 or on a Mac;
they're not even delivered via Unix I/O. Note that the thread on which
these events must be handled is the thread to which they are delivered
(and not by parrot's event loop): UI APIs are not thread-safe. So the UI
thread needs to be able to enter a parrot callback on the same thread.
Asynchronous I/O completion is surely considered less of an
event-handling problem and more of a thread-synchronization problem.
Also, parrot async I/O completion hopefully won't need to be serialized
through an I/O retirement thread ("event loop," whatever you want to
call it) except when the platform winds up requiring that through sloppy
async APIs.
This is really an asynchronous notification API, where asynchronous
completion is a very important "fires-once" form of asynchronous
notification. It's far, far below what programmers will expect when
hearing the term "event."
.NET's thread pool is a very close match to what you're discussing. A
.NET "WaitHandle" is almost identical to your event source. (Right down
to the waitone/waitany ops you just suggested.)
The intuitive concept of "events" isn't even in the same class of
problem that your document addresses. Probably best to use another term
for this (very cool, very necessary) technology.
--
Gordon Henriksen
IT Manager
ICLUBcentral Inc.
gor...@iclub.com
P.S. - Now that they're in, have you considered using objects and
methods instead of opcodes to define parrot APIs? If parrot technologies
are exposed through objects, you'll save on opcount, and HLLs won't need
to build yet another a shim for every new parrot feature. Dogfood.
Gordon Henriksen
> chromatic wrote:
>
> > So for SDL, I'd start a separate thread that blocks on
SDL_WaitEvent,
> > creating and posting events when they happen. My main program would
> > handle the events as normal Parrot events. Standard producer
consumer
> > stuff.
> >
> > Since it's blocking, it won't eat up too many resources --
> > that's nice. It'd be nice to have the SDL event thread ignore events
> > I don't care about though, instead of creating event PMCs I'll just
> > throw away later.
>
> You can always Get Horribly Clever in the event handling thread and
> look at what the SDL library's handed you. If it's uninteresting you
> can just throw it away rather than creating an event to be discarded.
>
> > Is this what you have in mind?
>
> Yep.
As I pointed out in another post, this doesn't work for integrating with
at least two significant "event sources:" Windows and the Mac OS. :) UI
events need to be handled synchronously on the thread to which they were
delivered, since the GUI APIs are not threadsafe. Trying to handle these
events from another thread is, quite simply, a doomed endeavour.
Uri Guttman
>> i disagree with the two classes. how can you handle an unexpected event
>> without know it could happen? effectively all events are expected, some
>> are more expected than others. :)
DS> The terminology there's a bit strained, and I think it's in large part
DS> responsible for most of the rest of the confusion.
that seems to be the case. do an edit pass with better names and clearer
definitions.
DS> They're probably better called Named and Anonymous events, though
DS> that's a bit dodgy in some ways too. Maybe specific and generic events
DS> would be better.
DS> The basic difference is that with Named events you *know* in advance
DS> that the event is coming because you explicitly requested it by
DS> setting a timer or making an IO request. (A specific timer or IO
DS> request, not timer or IO requests in general) This is different from
DS> things like signals as you've not asked for a particular signal to
DS> fire--they fire off whenever something outside your control and
DS> expectation happens.
ok. now signals are not normally 'handled' by any code (c included)
unless you specify something. so all events at that level are always
specified. if parrot needs to handle signals even when no user code
asked to handle them, then i can see a need for this separation. but it
should only be used for internal parrot stuff and all other events
should be specified at the user level. i see no reason for any user code
(PASM and up) to not do a proper handler specification.
DS> Because of this, you have the event PMC for a Named event before the
DS> event occurs and thus can wait on it. You *don't* have the event PMC
DS> for an anonymous event, so you can't wait on it, all you can do is
DS> semi-generically react once it's occurred.
i think that should read "all parrot can do is react, assuming it is the
one that set the signal handler (at the c/kernel level)".
DS> Callbacks and User Data
DS> =======================
>>
DS> Each expected event can have a callback routine and piece of user
DS> data associated with it. When the event request is satisfied (The IO
DS> operation completes or the timer fires, for example) the callback
DS> associated with it is invoked. The user data, as well as the event
DS> data, if there is any, is passed into the callback.
>>
>> the event handle(r) itself also is passed to the callback.
DS> No, it isn't.
i think i meant the event itself. the handler is what is called in the
callback.
>> you say event
>> data there and maybe you mean event structure/object?
DS> Nope, I mean event data. It's the data that whatever triggered the
DS> event generated.
then those terms need to be cleanly defined. user data is passed into
the event and passed back to the callback. it is meant for the user code
to associate an event with some user state. it can be an
object/structure/id//whatever. the event data is the actual data
returned by the event iself from the event source. it will contain data
read from an file/socket or mouse coords/stuff or signal
information, etc.
DS> Callback signatures are fixed, and of the form:
>>
DS> (Preturndata, Icommandstatus) =
DS> callbacksub(Pevent, Peventdata, Puserdata, Icommand)
>>
DS> The callback is passed in the event PMC, the PMC for the event data,
DS> and the PMC for the user data, if any. (Either or both of these can
DS> be the Null PMC if there is no user or event data) Command is always
DS> 0 for callback handling. (Callbacks and IO layer functions are
DS> identical. More detail in the IO section)
>>
>> how about renaming userdata to iodata since that is what it
>> is.
DS> It isn't though. The event data is the IO data here.
ok, i can work with your name choices as long as they are clearly defined.
>> huh? if you are in an event handler, you have seen the event (and are
>> presumably in user level code), so why would you wait for it again?
DS> You're confused here. The event handler is what throws the
DS> exception. It's the user code that's waited on the event that gets the
DS> exception. (And it's possible, though likely a bad idea, to have an
DS> exception handler make an async request and wait for it to complete)
i am confused everywhere! :)
the OP says this:
DS> With an expected event, if an event handler is terminated by an
DS> unhandled exception, that exception is attached to the event itself,
DS> and when user-level code C<wait>s on the event, the exception is
DS> thrown then.
you have to be in an event handler to terminate it. so you must have
waited for it or be in a callback. so why would you need to wait for it
again after it terminated because of an exception? you are in user code
space now so let the exception system do its thing (like you said). if
it isn't caught, do the normal thing. i don't see why the event needs to
be tagged with the exception. if it wasn't handled in the (already
called) handler environment why would another wait be any better?
my point is that an event handler IS user level code. all the event
system does is monitor the actual events, queue the triggered ones and
dispatch to handlers via callbacks. the handlers and exception handling
is all user space above that.
>> you also can use a 'stopped/disabled' state. in many cases you want
>> an active event but stopped for a while.
DS> That can't happen. You can cancel an event, but that's it. You can
DS> stop an event source, but that's different.
that is bad IMO. why do you need the entire overhead of
creation/destruction of an event just to stop it temporarily? this
buffered async write scenario is common and very useful. it is so much
easier and faster to just stop/start the existing event instead of
creating/cancelling a new one each time the writer needs to fire up.
this can be done easily for normal i/o events with a flag which makes
the select/poll loop skip it. if you are going to do blocking
reads/writes in individual kernal threads for this, then it will be
harder or impossible. you could still dedicate one thread to handle all
simple i/o event requests and have it run a select/poll loop for them
all. then when it gets a triggered event, it does the same queueing of
it that you want to do now. this way you save on threads, allow for
stop/start and still can use other kernel threads for special event
stuff that won't always work well with event loops (signals for one).
>> add enable/disable here.
DS> Nope. Events aren't enable-able/disable-able.
can you explain why not?
>> i don't think there is a need for all those variants. why would alarm
>> need any special opcode when it is just a timer with a delay of abs_time
>> - NOW? let the coder handle that and lose the extra op codes.
DS> I didn't see any reason to not do absolute times as well as
DS> deltas. It's no big deal either way.
>> also the interval can be folded in as a extra arg to the timer (either a
>> repeat flag or an interval). the initial seconds can be the first delay
>> and then reused for intervals or the interval value can take over:
>>
>> settimer Pevent, Idelay_seconds, Iinterval_seconds[, Pcallback,
>> Puserdata]
>>
>> if interval_seconds is set (>0), use it for a repeating timer. if
>> delay_seconds is set (>0), it is the first interval.
DS> Ick. No. It's not like there's any less code there, and the intent's a
DS> bit clearer.
but a combination of the two still has one nice feature. you can have
the initial delay be X and the interval afterwards be Y. not a deal
breaker and easily done with a wrapper but i like it. :)
DS> INTVAL Parrot_ping_event(Parrot_Interp Interpreter, INTVAL type);
>>
DS> Note that an event of type C<type> has just occurred. Returns either
DS> 0 on success or 1 on failure. This function may fail if the target
DS> interpreter is unable to post an event to its event queue. This
DS> normally happens because there are no event PMCs available to
DS> allocate for the event.
>>
>> clarify. who 'notes'?
DS> Posts an event of the specified type to the event queue of the target
DS> interpreter.
>> maybe call it 'check_event'? how is this different
>> than post_event?
DS> No PMC handling. It means that whatever code's doing it doesn't have
DS> to manage a PMC queue (since if it's in an interrupt handler it can't
DS> allocate one--they need to be preallocated) and set up all the bits of
DS> the event PMC. Of limited utility, but for things like signal handlers
DS> that can't do much anyway and have so little information available to
DS> them it's sufficient.
ok, so ping is a lightweight post? the spec should say something like
that mixed with your comments.
Uri Guttman
mab> You wrote:
>> i don't think there is a need for all those variants. why would alarm
>> need any special opcode when it is just a timer with a delay of abs_time
>> - NOW? let the coder handle that and lose the extra op codes.
mab> No, you don't want to do it that way. Becasue you want to make
mab> the latency between getting the abs_time, doing the substract and
mab> actually setting up the time as small, as possible you almost
mab> have to do this operation as a builtin op. In fact you can argue
mab> that you want to lock out async events while doing it as well.
i always assume timers to trigger after a specified delay. accuracy of
delivery (latency) is silly to worry about in perl for granularities of
more than about .05 seconds or so. building a very fine grained accurate
real time system in perl makes little sense to me. so i usually don't
worry about who does the delta calculation and the slight amount of
delay it takes. it will be much smaller than the typical granularity of
the timings anyway.
yes, i can see how an slow async event between the delta calc and the
actual event creation could cause a problem. a way around it is to use a
cronlike thing which has an interval timer and checks for the actual
absolute time and not some precalculated (and possibly way off) delta.
again, i assume event timers are minimum delay devices and i never
assume any sort of decent latency. you get called some time AFTER the
delay is over. so absolute times should be done with a repeat timer set
for the actual granularity you want. and most absolute timers are in 1
minute or more granularites (e.g. cron).
so i would never use the absolute timer in parrot since i couldn't trust
it any more than i can trust a repeated interval timer. but while i can
synchronize interval timers to the real clock, i can't adjust a
miscalculated absolute timer.
Leopold Toetsch
> None of this is set in stone, but take a look and see how everyone feels
> about this.
> I'll get to IO after, since the two are pretty tightly intertwined, and
> changes to this will affect the IO stuff too.
> ------------>Snip here<---------------
> Events, another design sketch
I don't have the time to go through it thorougly, but nethertheless here
are some remarks in no particular order:
1) Before discussing it, the document needs a glossary of all used terms
There's already enough confusion.
2) I think, the exception related part should just be postponed until
exceptions are layed out.
3) This "here are some opcodes" part isn't really needed at this stage.
(not to talk about 12 additional op variants, just to set a timer,
which is already possible now with *no* additional opcode)
The ops do and can't in no way cover all possible event's data.
You started off with "array/hash" combination PMC, which is good,
then that idea got lost and here's again the opcode explosion.
4) I don't think, that the *internal* event is already a PMC. You can't
create one in a signal handler (yes the term pre-creation was uttered
- how many do you pre-create?) PMCs can *only* be created in user
code. For signals and such, that's at event delivery just before a
PASM handler get's running.
5) "event level" jumps in at the end in the opcode section. See 1)
6) A lot of Uri's comments apply, e.g. event states ...
Enough for now
leo
Leopold Toetsch
> DS> Because of this, you have the event PMC for a Named event before the
> DS> event occurs and thus can wait on it. You *don't* have the event PMC
> DS> for an anonymous event, so you can't wait on it, all you can do is
> DS> semi-generically react once it's occurred.
> i think that should read "all parrot can do is react, assuming it is the
> one that set the signal handler (at the c/kernel level)".
I think, *if* we want an event PMC, we can alyways create one, when the
user code indicates that this kind of event should be handled.
$SIG{CHLD} = sub { 1; };
This could probably create the event PMC, associate the user callback
with it, enable SIGCHLD and be done with it. It's the same as with a
timer event.
*But* the first question is: do we really want PMCs in the inyards of
the event system (except callback sub PMCs)?
There are several issues:
- these PMCs need marking during DOD, which means that we have to mark
the event queue that might change under us
- event data may cross thread boundaries. This implies that event PMCs
(at least this kind) have to be shared PMCs (and all their contents)
Finally, if there is something like an "Unnamed Event" that needs
creating a PMC, we can't create the PMC in interrupt code. So the PMC
has to be preallocated. But how many PMCs will you provide? In which
interpreter's arena are these precreated?
For now I think, it's not a good idea to have the internals of an event
as a PMC.
leo
Leopold Toetsch
> settimer Pevent, Idelay_seconds, Iinterval_seconds[, Pcallback,
> Puserdata]
> so now all you have is two signatures (for float or integer seconds).
No. Above C<setttimer> definition expands to 8 different functions for
INTVAL arguments only:
Parrot_settimer_p_i_i
Parrot_settimer_p_i_i_p_p
Parrot_settimer_p_i_ic
Parrot_settimer_p_i_ic_p_p
Parrot_settimer_p_ic_i
Parrot_settimer_p_ic_i_p_p
Parrot_settimer_p_ic_ic
Parrot_settimer_p_ic_ic_p_p
When you use NULL PMCs for the optional arguments there are still 4.
This is "one" opcode only for one specific event. When we start creating
opcodes for each possible event, we'll have tons of additional opcodes.
This is really not needed.
This is the current and working code to program a timer:
new P1, .Array
set P1, 8
set P1[0], .PARROT_TIMER_NSEC
set P1[1], 0.2
set P1[2], .PARROT_TIMER_HANDLER
find_global P2, "_timer_sub"
set P1[3], P2
set P1[4], .PARROT_TIMER_REPEAT
set P1[5], 2
set P1[6], .PARROT_TIMER_RUNNING
set P1[7], 1
new P0, .Timer, P1
This doesn't use any additional opcodes and is compliant with:
$ perldoc -F docs/pdds/pdd02_vtables.pod
/init_pmc
This scheme is expandable too. The parameters that an event might need
aren't limited and permutations don't hurt.
> uri
leo
Dave Whipp
> The terminology there's a bit strained, and I think it's in large
> part responsible for most of the rest of the confusion.
>
> They're probably better called Named and Anonymous events, though
> that's a bit dodgy in some ways too. Maybe specific and generic
> events would be better.
I've always distinguished these as "solicited" and "unsolicited" events,
assuming I'm understanding you correctly..
Dave.
Dan Sugalski
>Dan Sugalski wrote:
>
>> chromatic wrote:
>>
>> > So for SDL, I'd start a separate thread that blocks on
>SDL_WaitEvent,
>> > creating and posting events when they happen. My main program would
>> > handle the events as normal Parrot events. Standard producer
>consumer
>> > stuff.
>> >
>> > Since it's blocking, it won't eat up too many resources --
>> > that's nice. It'd be nice to have the SDL event thread ignore events
>
>> > I don't care about though, instead of creating event PMCs I'll just
>> > throw away later.
>>
>> You can always Get Horribly Clever in the event handling thread and
>> look at what the SDL library's handed you. If it's uninteresting you
>> can just throw it away rather than creating an event to be discarded.
>>
>> > Is this what you have in mind?
>>
>> Yep.
>
>As I pointed out in another post, this doesn't work for integrating with
>at least two significant "event sources:" Windows and the Mac OS. :) UI
>events need to be handled synchronously on the thread to which they were
>delivered, since the GUI APIs are not threadsafe.
Oh, it's worse than that--GUI commands need to be issued from the
main thread, at least with OS X. (There's no actual requirement as to
which thread handles the actual events as long as you treat the OS
event queue as the thread-unsafe thing it seems to be) Or so the docs
seem to indicate, though they may be a bit conservative.
Dan Sugalski
Well, it could work, but it'd take an awful lot of effort and it's
possibly a bad idea. Or... not.
But yes, we may be stuck with two event loops (or a master/slave loop
setup) in some places.
>If you're calling this an event system, it ought to mesh in with the
>notion of events that every programmer who uses them will have:
>
> - Mouse events.
> - Keyboard events.
> - Redraw events.
> - Drag-'n'-drop events.
> - Menu command events.
> - etc., etc., etc..
And it does. Or, rather, it can, and it will many places, though not
all of them, unfortunately.
>You won't get any of these events from an I/O wait on Win32 or on a Mac;
>they're not even delivered via Unix I/O. Note that the thread on which
>these events must be handled is the thread to which they are delivered
>(and not by parrot's event loop): UI APIs are not thread-safe. So the UI
>thread needs to be able to enter a parrot callback on the same thread.
That's not necessarily true, actually, at least not for OS X. (I
can't speak for Win32)
>Asynchronous I/O completion is surely considered less of an
>event-handling problem and more of a thread-synchronization problem.
>Also, parrot async I/O completion hopefully won't need to be serialized
>through an I/O retirement thread ("event loop," whatever you want to
>call it) except when the platform winds up requiring that through sloppy
>async APIs.
>
>This is really an asynchronous notification API, where asynchronous
>completion is a very important "fires-once" form of asynchronous
>notification. It's far, far below what programmers will expect when
>hearing the term "event."
Well... maybe. For many programmers it's dead-on what they're
expecting. If you're writing server or console code, or dealing with
quite a number of the 'cross-platform' GUI APIs, it's exactly what
people expect. For Win32 and OS X programming, it's less so, but in
that case you have to give yourself over to the local event loop.
*Something* has to run the show, and in that case it's pretty clear
that it has to be the OS' event loop.
There is a reasonably simple way around this (And with a bit of
trickery I could do it for OS X) which is to have the OS callbacks do
nothing but post events into Parrot's event queue and leave it to
parrot to then manage the (now local) events and do all the Clever
Things that your average GUI program needs to do. Whether this is
feasible or not is somewhat up in the air, though. (Objective-C makes
this easy. Can't say how it works other places)
>The intuitive concept of "events" isn't even in the same class of
>problem that your document addresses. Probably best to use another term
>for this (very cool, very necessary) technology.
I think that very much depends on both the system you're working on
and your concept of events.
>P.S. - Now that they're in, have you considered using objects and
>methods instead of opcodes to define parrot APIs? If parrot technologies
>are exposed through objects, you'll save on opcount, and HLLs won't need
>to build yet another a shim for every new parrot feature. Dogfood.
I have, but I don't really want to pay the overhead, and since there
are enough shims that'll have to be in place as it is I'm not too
concerned. Once you're past three or four you just end up building a
general-purpose table-driven translator to sort out what's an op,
what's a sub, and what's a method on an object. (Well, OK, you don't
have to, but then things get hellish pretty quickly)
OTOH it's probably about time for me to get over my dislike of
objects and just deal with it. Timers'd be a good place to start, I
suppose.
Gordon Henriksen
> At 2:59 PM -0400 5/11/04, Gordon Henriksen wrote:
>
>> As I pointed out in another post, this doesn't work for integrating
>> with at least two significant "event sources:" Windows and the Mac
>> OS. :) UI events need to be handled synchronously on the thread to
>> which they were delivered, since the GUI APIs are not threadsafe.
>
> Oh, it's worse than that—GUI commands need to be issued from the main
> thread, at least with OS X. (There's no actual requirement as to which
> thread handles the actual events as long as you treat the OS event
> queue as the thread-unsafe thing it seems to be) Or so the docs seem
> to indicate, though they may be a bit conservative.
That's exactly true, and exactly my point. GUI APIs are definitely not
thread-safe, and for very good reasons. Not only do the system APIs
essentially mandate it, but any depth of thought will make it obvious
that UI events must be handled synchronously. Keeping also in mind that
the main event loop stays on the execution stack, waking up to call
back into event handlers; WaitNextEvent is dead. So funneling all event
delivery through a Big Parrot Queue is unfeasible if your definition of
"event" includes UI events. UI events, with their threading and context
requirements, are really much more an issue of NCI callbacks,
re-entrancy, and wrappers around system types.
That said, the technology you're proposing is still, exactly as it
stands, incredibly useful:
• Asynchronous completion.
• Signal handling.
• Runtime state change notifications, for undoing speculative
optimizations.
It's just not at all suitable for handling GUI events. Which is
perfectly fine: Just either
1. Don't call it "events" so that people aren't disappointed and
frustrated, or
2. Figure out some way to fold in GUI events.
—
Dan Sugalski
Yep, that works too. (I think I had that in an earlier draft but
changed it. I'm not sure my change was for the better...)
Brent 'Dax' Royal-Gordon
>> Oh, it's worse than that—GUI commands need to be issued from the main
>> thread, at least with OS X. (There's no actual requirement as to which
>> thread handles the actual events as long as you treat the OS event
>> queue as the thread-unsafe thing it seems to be) Or so the docs seem
>> to indicate, though they may be a bit conservative.
> 1. Don't call it "events" so that people aren't disappointed and
> frustrated, or
> 2. Figure out some way to fold in GUI events.
One general-purpose Parrot event queue, plus another for the GUI thread.
When you call a method on the GUI bindings, it really enqueues an
event for the GUI thread, which receives the message and calls the
actual method.
Or for that matter, enqueue an event in the main queue that only the GUI
thread is interested in.
PITA? Yeah. But it oughta work, and if that's the hard part of writing
your GUI binding library, you should consider yourself lucky.
--
Brent "Dax" Royal-Gordon <br...@brentdax.com>
Perl and Parrot hacker
Oceania has always been at war with Eastasia.
Dan Sugalski
>Uri Guttman <u...@stemsystems.com> wrote:
>> >>>>> "DS" == Dan Sugalski <d...@sidhe.org> writes:
>
>> DS> Because of this, you have the event PMC for a Named event before the
>> DS> event occurs and thus can wait on it. You *don't* have the event PMC
>> DS> for an anonymous event, so you can't wait on it, all you can do is
>> DS> semi-generically react once it's occurred.
>
>> i think that should read "all parrot can do is react, assuming it is the
>> one that set the signal handler (at the c/kernel level)".
>
>I think, *if* we want an event PMC, we can alyways create one, when the
>user code indicates that this kind of event should be handled.
>
> $SIG{CHLD} = sub { 1; };
>
>This could probably create the event PMC, associate the user callback
>with it, enable SIGCHLD and be done with it. It's the same as with a
>timer event.
Which is swell, except.... the problem you run into here is
persistent signal handlers. (Which you really do need, or you lose
signals, which is bad) This goes for GUI event handlers too. You can
pre-allocate a PMC, but then you need to make sure the
>*But* the first question is: do we really want PMCs in the inyards of
>the event system (except callback sub PMCs)?
We need something. Not having a PMC doesn't make things any better,
since the problems persist regardless of what we might want to do. If
something can fire more than once it needs to have something to
inject into the event queue, and that something can't be allocated
from the OS generally (memory allocation's not callable from
interrupt level)
No matter what we do parrot's going to have to maintain a pool of
these things. Making it a PMC means we've got garbage collection to
clean up after them, which is at least something.
>There are several issues:
>- these PMCs need marking during DOD, which means that we have to mark
> the event queue that might change under us
>- event data may cross thread boundaries. This implies that event PMCs
> (at least this kind) have to be shared PMCs (and all their contents)
Yep. This is a definite possibility, though it depends on how we
handle things. If the only threading stuff involved is at the C level
(ie the thread code doesn't execute any parrot bytecode) then it
doesn't much matter.
>Finally, if there is something like an "Unnamed Event" that needs
>creating a PMC, we can't create the PMC in interrupt code. So the PMC
>has to be preallocated. But how many PMCs will you provide? In which
>interpreter's arena are these precreated?
Very good questions. We have to do it, there's no choice. I left the
underlying mechanism open for discussion.
Leopold Toetsch
> At 9:41 AM +0200 5/12/04, Leopold Toetsch wrote:
>> $SIG{CHLD} = sub { 1; };
>>
>>This could probably create the event PMC, associate the user callback
>>with it, enable SIGCHLD and be done with it. It's the same as with a
>>timer event.
> Which is swell, except.... the problem you run into here is
> persistent signal handlers.
A persistent signal handler is some storage in the interpreter, that
associates the interrrupt number with a Sub PMC (or a chain of these).
The signal internally is a signal event. On delivery of the event (and
here we are already again in user mode) a PMC is created that holds the
event information. This PMC is then passed on to the users callback.
> ... This goes for GUI event handlers too. You can
> pre-allocate a PMC, but then you need to make sure the
[ sentence not finished but ... ]
... that you don't run out of pre-allocated PMCs. Which is likely when
you start tracking mouse move events or such. This doesn't work. You are
gonna loosing events. GUI events are coming from the GUI thread. You
can malloc an internal event structure holding the mouse coordinates,
but you can't create a PMC there.
Then the event gets put into Parrot's event system, then see below...
Signals or GUI events aren't really differing.
>>*But* the first question is: do we really want PMCs in the inyards of
>>the event system (except callback sub PMCs)?
> We need something. Not having a PMC doesn't make things any better,
> since the problems persist regardless of what we might want to do. If
> something can fire more than once it needs to have something to
> inject into the event queue, and that something can't be allocated
> from the OS generally (memory allocation's not callable from
> interrupt level)
I know that. The current scheme is safe WRT these problems. A signal
originates from the signal handler, incrementing a sig_atomic_t variable
per signal. Then the signal handler returns. No allocation of anything
is in the interrupt code.
But the io_thread (which is the only one that has the signal unblocked)
gets interrupted with EINTR. Here the (malloc) allocation of the event
structure is safe, and all needed stuff is put into the event structure.
The event is now posted to the global event thread which waits on the
queue condition.
In the event thread the signal gets broadcasted to all running
interpreters. And when it's in their task_queue, its fine to create an
appropriate event PMC, if that interpreter is interested in that event.
The signal delivery is the most complex and dangerous one. And it's
already working.
You can't allocate some PMCs easily, when you don't know, where they
will go to. This has nasty issues with DOD when running multiple
threads.
> No matter what we do parrot's going to have to maintain a pool of
> these things. Making it a PMC means we've got garbage collection to
> clean up after them, which is at least something.
Making it a PMC in the interpreter that gets the event ... means ...
>>Finally, if there is something like an "Unnamed Event" that needs
>>creating a PMC, we can't create the PMC in interrupt code. So the PMC
>>has to be preallocated. But how many PMCs will you provide? In which
>>interpreter's arena are these precreated?
> Very good questions. We have to do it, there's no choice. I left the
> underlying mechanism open for discussion.
There's always a choice. Please have a look at the current code.
leo
Dan Sugalski
>Dan Sugalski <d...@sidhe.org> wrote:
>> At 9:41 AM +0200 5/12/04, Leopold Toetsch wrote:
>
>>> $SIG{CHLD} = sub { 1; };
>>>
>>>This could probably create the event PMC, associate the user callback
>>>with it, enable SIGCHLD and be done with it. It's the same as with a
>>>timer event.
>
>> Which is swell, except.... the problem you run into here is
>> persistent signal handlers.
>
>A persistent signal handler is some storage in the interpreter, that
>associates the interrrupt number with a Sub PMC (or a chain of these).
>The signal internally is a signal event. On delivery of the event (and
>here we are already again in user mode) a PMC is created that holds the
>event information. This PMC is then passed on to the users callback.
Which doesn't address the problem. You can't create a PMC inside the
signal handler. Nor can you create anything *else* in the signal
handler. All you can do is use something that already exists.
Allocating a single structure to the signal handler doesn't help
since if a second signal comes in before the first is done (or
potentially even started) you'll lose it.
> > ... This goes for GUI event handlers too. You can
>> pre-allocate a PMC, but then you need to make sure the
>
>[ sentence not finished but ... ]
>... that you don't run out of pre-allocated PMCs. Which is likely when
>you start tracking mouse move events or such. This doesn't work. You are
>gonna loosing events. GUI events are coming from the GUI thread. You
>can malloc an internal event structure holding the mouse coordinates,
>but you can't create a PMC there.
>Then the event gets put into Parrot's event system, then see below...
>Signals or GUI events aren't really differing.
The GUI thread, if its in a position to malloc, is in a position to
allocate a PMC from the destination interpreter. That's not the
problem. The problem comes in when you're in interrupt mode and can't
allocate anything.
> >>*But* the first question is: do we really want PMCs in the inyards of
>>>the event system (except callback sub PMCs)?
>
>> We need something. Not having a PMC doesn't make things any better,
>> since the problems persist regardless of what we might want to do. If
>> something can fire more than once it needs to have something to
>> inject into the event queue, and that something can't be allocated
>> from the OS generally (memory allocation's not callable from
>> interrupt level)
>
>I know that. The current scheme is safe WRT these problems. A signal
>originates from the signal handler, incrementing a sig_atomic_t variable
>per signal.
That doesn't work. Besides requiring an aligned atomic word per
interrupt source, it requires something to scan those words
>But the io_thread (which is the only one that has the signal unblocked)
>gets interrupted with EINTR. Here the (malloc) allocation of the event
>structure is safe, and all needed stuff is put into the event structure.
>The event is now posted to the global event thread which waits on the
>queue condition.
That's pretty horribly platform-dependent. As schemes go... it can't
work most places. This is going to break badly on quite a few
platforms, especially once we start spawning more threads off.
> > Very good questions. We have to do it, there's no choice. I left the
>> underlying mechanism open for discussion.
>
>There's always a choice. Please have a look at the current code.
I have. It's fundamentally broken pretty much everywhere. We just
don't trigger off the broken behaviour yet.
Gordon Henriksen
> Gordon Henriksen wrote:
>
> >> Oh, it's worse than that—GUI commands need to be issued
> from the main
> >> thread, at least with OS X. (There's no actual requirement
> as to which
> >> thread handles the actual events as long as you treat the OS event
> >> queue as the thread-unsafe thing it seems to be) Or so the
> docs seem
> >> to indicate, though they may be a bit conservative.
> ...
> > 1. Don't call it "events" so that people aren't
> disappointed and
> > frustrated, or
> > 2. Figure out some way to fold in GUI events.
>
> One general-purpose Parrot event queue, plus another for the
> GUI thread.
> When you call a method on the GUI bindings, it really enqueues an
> event for the GUI thread, which receives the message and calls the
> actual method.
>
> Or for that matter, enqueue an event in the main queue that
> only the GUI thread is interested in.
Brent,
That's tantamount to fine-grained locking on a collection; instead of a
mutex, an OS thread does the serialization. Fine-grained locking doesn't
work.
It's also going to be ridiculously slow.
Can you imagine what havoc this would wreak with multiple "writers"? Not
a solution.
The Carbon Event Manager is (synchronously) interested in your
callback's return value; if a Carbon event handler callback returns
kEventNotHandled, the Event Manager may supply a default implementation.
Not a solution. A GUI event simply MUST be handled from the context that
the operating system or GUI toolkit called the callback; anything else
is addled.
This might be too large a problem, despite seemingly easy to squash into
one API at the first pass. Might it help to consider breaking down and
decoupling the problem for better flexibility? Here's a pass...
-> class: EventData
Parameter block storing information about an event that has
occurred. Passed as the only argument to event handlers by
convention. No intrinsic fields or operations.
Subclasses: IOEventData,
SignalEventData,
MacMouseEventData,
Win32MouseEventData,
...
-> class: Event
A registery of event handlers for a particular event. Defines
ordering and fallback behavior and soforth, should multiple
handlers be registered.
Instantiate one Event for each distinguishable kind of event
which can occur; handlers can then.
Anonymous handlers? Not so much. How magic collections which
make Events on-demand?
Methods: RegisterHandler(&handler)
UnregisterHandler(&handler)
This API is awkward without method pointers.
So add a userdata parameter; whatever, details.
CallHandlers(*@args)
Synchronously calls registered handlers.
Parrot's event loop would call this, as would
anything which wanted to control the thread on
which events were dispatched.
Subclasses? Maybe to override the behavior of CallHandlers,
for specialized fallback behaviors. Or perhaps to register
"interest" in the event with the OS event.
-> class: EventSource
Something that can wake parrot's main event loop.
Methods: Enable()
Turn on the spigot: Register the event source
with parrot's main event loop.
Disable()
Turn the spigot back off. Non-recurring sources
(e.g., one-off timers) ought to disable themselves
automatically.
Subclasses: Timer,
IOEventSource,
SignalEventource
This is all "user-mode" API, very close to the operations I'd want to
see from an HLL. Adapt for C/parrot core use as appropriate. I've
avoided cluttering the API by adding state variables; method pointers
generally take care of that just fine.
If EventData is kept naïve of Event, and Event is kept naïve of
EventSource, then I think this comprises a pretty agile design.
All a GUI event loop needs to do to fit in is to invoke
CallHandlers(...) on an Event from a C shim. The event will be handled
synchronously on that thread, and any return values necessary can be
passed back in a field of the EventData. (Parameter blocks are good for
both directions, after all.) Since the event loop would know the
thread/interpreter, it can safely create the EventData object.
For high-performance subsystems, it might be best to be able to skip the
EventData if no parameters to the operation are necessary.
Also consider user-mode events. A user program might want to use events
to, say, signal changes in data to UI views which are displaying that
data. That's probably a synchronous, rather than asynchronous,
operation.
All of the scary asynchronous stuff is hidden away in the EventSource
and parrot's event loop. Nothing above is so intrinsically asynchronous.
Which is fine.
The internal event queue and runloop undoubtedly require considerable
platform specialization and otherwise voodoo; "enabling" the
SignalNotificationSource is undoubtedly more complex than just that.
Implementation detail. They don't even appear in the design above; all
user-level. (Which is not to say that these details are unnecessary.)
Hope that's useful.
--
Leopold Toetsch
>>I know that. The current scheme is safe WRT these problems. A signal
>>originates from the signal handler, incrementing a sig_atomic_t variable
>>per signal.
> That doesn't work.
??? It's one of the few safe actions what you can do in an interrupt
handler. WTF should that not work.
> ...Besides requiring an aligned atomic word per
> interrupt source, it requires something to scan those words
Why aligned? If you can't set a sig_atomic_t var in a signal handler
and test that outside, you are hosed anyway. What is the problem with
that? How do you do it? The io_thread wakes up on EINTR and tests the
sig_atomic_t sighup or such, what's wrong?
> That's pretty horribly platform-dependent.
Of course. I did describe signals. They are platform-dependent. Win32
don't have much (and no usable) signals at all. This scheme is for
unixish plaftforms. Win32 will have a different one. OS X maybe too.
This is an implementation of events on a specific platform.
The whole event system is platform-dependent in the guts.
>>There's always a choice. Please have a look at the current code.
> I have. It's fundamentally broken pretty much everywhere. We just
> don't trigger off the broken behaviour yet.
This "pretty much everywhere" is e.g.: Thanks for your detailed
analysis.
leo
Rocco Caputo
> At 1:10 PM -0400 5/11/04, Uri Guttman wrote:
>
> >i don't think there is a need for all those variants. why would alarm
> >need any special opcode when it is just a timer with a delay of abs_time
> >- NOW? let the coder handle that and lose the extra op codes.
>
> I didn't see any reason to not do absolute times as well as deltas.
> It's no big deal either way.
You may need the distinction later.
POE's mailing list has a thread with a periodicity of about six
months: What happens when daylight-saving time comes or goes? The
answer: All your short-term timers fire at once if the clock moves
ahead, or everything stalls for an hour (or so) when it moves back.
So far the best solution requires a distinction between absolute and
relative timers: Adjust the relative timers to compensate for the
shift, but leave the absolute ones alone.
Relative timers tend to be used for things like timeouts and periodic
polling. You don't want all your TCP connections to suddenly time out
when the clock jumps ahead, and you don't want your log tailers to
pause an hour when it skips back. However, you usually do want your
absolute-timed jobs to run when the wall clock says they should be.
Something to keep in mind.
-- Rocco Caputo - http://poe.perl.org/
mar...@kurahaupo.gen.nz
> >> Why would alarm need any special opcode when it is just a timer
> >> with a delay of [abs_time minus NOW]?
> >> Let the coder handle that and lose the extra opcodes.
>
> mab> you want to make the latency between getting the abs_time, doing
> mab> the substract[ion] and actually setting up the time as small as
> mab> possible
>
> Accuracy of delivery (latency) is silly to worry about in Perl for
> granularities of more than about .05 seconds or so. Building a very fine
> grained accurate real-time system in Perl makes little sense to me.
> so i usually don't worry about who does the delta calculation and the
> slight amount of delay it takes.
Never mind the granularity or latency, there are systems where "time of day"
can be adjusted to take into account clock drift, while "system elapsed
time" is left unaffected. Which you want depends on whether you want to
sleep for a specific time, or wake up at a specific time, and it would be
nice if Parrot didn't rule out making use of that.
-Martin
Dan Sugalski
Yeah, this is a good point. Interval and time-of-day timer events
will stay separate. (And PMCs rather than ops, but that's a separate
thing :)
--
Dan
--------------------------------------it's like this-------------------